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Summary 



Having assessed various documents and oral evidence through a site visit, the 
Panel is convinced that NVAO acts in compliance with the ENQA membership 
regulations and is in substantial compliance with the Standards and Guidelines 
for Quality Assurance in the European Higher Education Area. The Panel 
therefore recommends to the Board of ENQA that NVAO's Full Membership of 
ENQA be confirmed for a further period of five years. 

The Panel concludes that NVAO fully complies with the ENQA membership criteria 
2, 3, 4, 5, 6, 7 and 8 and substantially complies with the ENQA membership 
criterion 1. 

A list of the recommendations of the Panel is provided in section 5.2 but, in 
short, the main areas for further improvements relates to public information in 
various senses and to the formulation of explicit and public criteria about how 
NVAO reaches its accreditation decisions in relation to the programme 
assessments in all cases. 

During the site visit, interviewees mentioned several times that NVAO is in a 
"transition state" as models and procedures for quality assurance are currently 
being changed in relation to Flanders or have recently been changed in the 
Netherlands. It is the view of the Panel that a situation of change and 
development is to be expected for most quality assurance agencies. The Panel 
recognises that NVAO and its partners have work ahead in order to establish the 
institutional assessment approach in Flanders as well as to consolidate the new 
practice in the Netherlands. The Panel encourages all the parties to learn from 
the experience gathered thus far. 



3 



1 Background and outline of the review process 



ENQA's regulations require all full member agencies to undergo an external 
cyclical review, at least once every five years, in order to verify that they fulfil 
the membership provisions. In November 2004, the General Assembly of ENQA 
agreed that the third part of the Standards and Guidelines for Quality Assurance 
in the European Higher Education Area (ESG) should be incorporated into the 
membership provisions of its regulations. Substantial compliance with the ESG 
thus became the principal criterion for Full membership of ENQA. 

This is the report of the review of NVAO undertaken in June 2012 for the purpose 
of determining whether the agency meets the criteria for full membership of 
ENQA. The process in general, including the structure of the present report, was 
run in accordance with the Guidelines for external reviews of quality assurance 
agencies in the European Higher Education Area. 

The present review of NVAO has been coordinated by ENQA. This is the second 
external review of NVAO according to the ESG. The first review took place in 
2007. 

The Panel for the 2012 external review of NVAO, appointed by ENQA, was 
composed as follows: 

• Tove Blytt Holmen, Director of the Department of Quality Assurance, NOKUT, Norway (Chair) 

• Signe Ploug Hansen, Director of Methodology, EVA, Denmark (Secretary) 

• Julian Tobias Hiller, Student at Leibniz University of Hannover, Germany 

• Elie Milgrom, Emeritus professor, consultant in professional development of teaching staff, 
Belgium 

• Dr. Norma Ryan, Director of the Quality Promotion Unit, University College Cork, Ireland 

The Panel was provided with ENQA's Terms of Reference (ToR), including a 
suggested timeline (see annex A). A short presentation of the Panel members is 
provided in annex B. 

NVAO produced a self-evaluation report (SER) which, together with the 
attachments to it, provided a substantial portion of the evidence that the Panel 
used for its analysis and to reach its conclusions. The Panel conducted a site visit 
to validate the statements made in the self-evaluation report, to clarify any 
points at issue, and to obtain further relevant documents. For an overview of 
assessed documents, see annex C. Before finalising the report, the Panel 
provided an opportunity for NVAO to comment on the factual accuracy of the 
draft report. 

The Panel confirms that it was given access to all the documents it requested and 
to all people it wished to consult with throughout the review. 
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2 Glossary of acronyms 

ECA: European Consortium for Accreditation in Higher Education 
ENQA: European Association for Quality Assurance in Higher Education 
EQAR: European Quality Assurance Register for Higher Education 
EVA: The Danish Evaluation Institute 

ESG: Standards and Guidelines for Quality Assurance in the European 
Higher education Area 

HEI: Higher Education Institution 

NOKUT: Norwegian Agency for Quality Assurance in Education 

NVAO: The Accreditation Organisation of the Netherlands and Flanders 

QANU: Quality Assurance Netherlands Universities 

SER: Self-evaluation report 

ToR: Terms of Reference 

VLHORA: The Flemish Council of University Colleges 
VLIR: Flemish Interuniversity Council 
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3 Introduction 



i. Purpose(s) of the review 

In its SER dated April 16 th 2012, NVAO describes its "main objectives" for the 
review as follows: 

1. Renewal of NVAO's full membership of ENQA and extended EQAR-registration. 

2. A first external reflection on the new accreditation system in the Netherlands. 

3. Contribution to NVAO's internal quality monitoring. 

ENQA's ToR, dated February 2012, identifies the review of NVAO to be a type A 
review. However, the ToR also stated: "In addition to ... the review aims to give a 
first external reflection on the new accreditation system in the Netherlands. " 

When asking ENQA for clarification on its mandate, the Panel received the 
following answer: "... NVAO would like to know from the Panel whether the new 
system is fit for the purposes NVAO wants to achieve, in accordance with the 
ESG/ENQA membership criteria and which advice could be given to improve it ..." 

Furthermore: "... There are no particular expectations on the Panel for n.3 as this 
is not part of the purposes agreed between ENQA and NVAO. ... what is meant by 
"Contribute to NVAO's internal quality monitoring" is that the overall result of the 
review, all Panel's recommendations will contribute to the agency's quality 
monitoring. " 

The Panel understands its mandate was to conduct a type A review where the 
focus has to be upon the current status and not upcoming developments (as per 
"Guidelines for external reviews ..."), i.e. it should investigate programme 
accreditation in Flanders and the new approach in the Netherlands combining 
institutional audit and programme accreditation. The Panel's reflections around 
the new system in the Netherlands will concentrate upon whether the system 
supports the ESG or whether it incorporates elements that are in contradiction 
with the ESG. 

The Panel finds it important to point out that it has decided to rely upon the 
assessments made in the first ENQA review of NVAO in 2007 in relation to those 
general characteristics that have not changed between 2007 and 2012. 

Being the accreditation organisation for both Netherlands and Flanders, NVAO is 
the body which makes all the accreditation decisions. However, the assessments 
of existing programmes on which the programme accreditation decisions of 
NVAO are based are carried out by other quality assurance bodies. Some of 
these are members of ENQA as a result of evaluations similar to the present one. 
The Panel has noted that VLHORA, VLIR and QANU have all undergone 
independent external reviews (in 2008, 2009 and 2010 respectively) and have 
been reviewed specifically with regard to their alignment and compliance with the 
ENQA membership criteria and thus the relevant sections of the ESG. The Panel 
has decided not to re-visit these assessments or to question the evidence 
presented in those assessments but to accept the decisions of the review panels 
that conducted those assessments. Thus in those topics of relevance to this Panel 
in its review of NVAO, the Panel took as read the compliance of these agencies 
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and accepted the credentials and expertise of these agencies in relation to the 
quality assurance activities undertaken. 

As regards the relation between NVAO and the quality assurance agencies the 
Panel finds it important to note that the agencies are not perceived as 
subcontractors to NVAO neither by NVAO, nor by the agencies themselves, nor 
by the Panel. In various dictionaries a subcontractor is generally defined as an 
individual or in many cases a business that signs a contract to perform a part or 
all of the obligations of another's contract. In its decision making of existing 
programmes, NVAO has to build upon quality assurance performed by other 
agencies as regulated by law and the assessments undertaken by the agencies 
are thus not based on a contract with NVAO. In case of assessments of new 
programmes and institutional assessments NVAO performs external quality 
assurance by its own panels. 

ii. The place of NVAO in the quality assurance structure of its 
jurisdiction 

This subsection and subsection iii are largely based on the background 
information provided in the SER of NVAO. All external stakeholders interviewed 
during the site visit expressed the view that the SER provides a reliable and 
adequate presentation of NVAO and the systems of external quality assurance in 
the Netherlands and Flanders and the Panel is thus confident that the information 
provided in the SER is fully reliable. 

NVAO, the Accreditation Organisation for the Netherlands and Flanders 
[Nederlands-Vlaamse Accreditatieorganisatie] was formally established by the 
Dutch and Flemish governments as a bi-national organisation on February l st 
2005. NVAO is thus the official, public and bi-national accreditation organisation 
in the Netherlands and Flanders. NVAO covers all types of higher education 
institutions (HEI) within its geographic domain. By law accreditation is the model 
for quality assurance. 

The primary assignment of NVAO is defined in the Treaty between the Dutch and 
Flemish Education ministers of 2003, which is described as: 

The primary assignment of NVAO is to accredit existing programmes in higher 
education (accreditation) and to assess new programmes (initial accreditation) in 
the Netherlands and Flanders. NVAO can be requested to carry out additional 
tasks by the Ministers of higher education in both countries if these assignments 
support or supplement NVAO's primary assignment. 

In the Netherlands, the first accreditation system was operational from 2003 to 
2010, in Flanders from 2005 to 2012. So the Netherlands has already made the 
transition to the new system and it is expected that Flanders will follow in 2013 
for programme accreditation, while implementing institutional reviews in 2015. 

Until the end of 2010, the accreditation system both in the Netherlands and 
Flanders was based exclusively on (initial) programme accreditation. In 
Flanders, this system is still current, at least until the end of the academie year 
2012 - 2013 (what will happen later depends on pending legislation). Since 
January l st 2011, a new accreditation system based on institutional audit 
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combined with (initial) programme accreditation was introduced in the 
Netherlands. 

iii. The main functions of NVAO, its current main areas of 

responsibility and work, including the review methods it uses 

(Initial) programme accreditation 

Programme assessment and accreditation is the core of both the accreditation 
system still in place in Flanders and the new system in the Netherlands. 

The framework for assessments of existing programmes still in place in Flanders 
comprises six themes. The six themes are subdivided into 21 standards that are 
assessed on a four-point scale (excellent, good, satisfactory and unsatisfactory). 
For assessing at theme level a dichotomous scale is used (satisfactory or 
unsatisfactory), as is the case for the final assessment. A very similar 
assessment framework is applied for accreditation of new programmes. In case 
of negative decisions of existing programme accreditation a recovery period of 
maximum 3 years is possible. 

In the new system in place in the Netherlands the final judgement of a 
programme is based on a four-point scale (Excellent, Good, Sufficiënt and 
Insufficiënt); this scale is applicable both on the level of a study programme and 
on the level of the separate standards of the framework. The number of 
standards depends on the framework applied (cf. below) 

The new system for programme accreditation in the Netherlands contains a 
recovery period and an elaborated procedure for initial accreditation; if a new 
programme is approved, NVAO can restrict the duration of the initial 
accreditation and subject it to conditions that have to be met within one year. In 
case of accreditation the recovery period is limited to two years. 

The new institutional audits in the Netherlands 

The new institutional audits in the Netherlands concern the assessment of an 
institution's quality assurance system and ultimately lead to a judgment about 
whether an institution is in control of the quality of its education programmes. 

These audits comprise five standards and can have three possible outcomes: 
satisfactory, conditionally satisfactory and unsatisfactory. A final conclusion that 
is either 'satisfactory' or 'conditionally satisfactory' means that the assessment of 
the programmes follows the framework of the so-called 'limited programme 
assessment'. This framework contains only three standards. 

If an institution fails the institutional audit or if it does not want to participate in 
it for specific reasons (for example, the limited size of an institution), the 
programmes will be assessed on the basis of the framework of the so-called 
'extensive programme assessment' which contains more standards than the 
framework of 'limited programme assessment' and strongly resembles the 
framework for programme assessment in place in the Netherlands before 2011 
and still in place in Flanders. 

The role of NVAO 

NVAO performs all processes related to the institutional audits in the Netherlands 
and initial programme accreditation in both the Netherlands and Flanders. NVAO 
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also has the authority to make all the accreditation decisions for existing 
programmes, both in Flanders and in the Netherlands. Accreditation decisions on 
existing programmes are made upon quality assessments and reports from 
quality assurance agencies, some of which are also ENQA-members. 

The role of NVAO, the division of labour between NVAO and the quality assurance 
agencies and the details of the review methods used are presented in various 
sections of chapter 4. 

iv. NVAO's engagement with the ENQA membership provisions/ ESG 

In its assessment frameworks, NVAO explicitly states that its standards have 
been developed in accordance with the ESG. 

NVAO also has an active internationalization policy and real engagement 
regarding higher education and quality assurance promoting ENQA, ECA and ESG 
in many ways. 

v. Methods employed by the Panel 

The Panel was appointed in May 2012 and received the Self Evaluation Report 
(SER) of NVAO at the same time. The SER and its attachments were analysed by 
each of the Panel members before the site visit and the Panel discussed the 
individual findings during its preparatory meeting, the day before the site visit. 
The site visit took place on June 26 th and 27 th 2012 in the NVAO offices in The 
Hague. NVAO drew up the programme for the site visit in close cooperation with 
the chair and secretary of the Panel. The programme included interview sessions 
with 36 people, including the chairman and members of the NVAO Executive 
Board and General Board, the NVAO managing director and a number of staff 
members, representatives of the Advisory Council of NVAO, representatives of 
umbrella organisations of higher education institutions, representatives of 
student organisations, representatives of quality assurance agencies and 
representatives of Dutch and Flemish ministries of higher education. The 
management of NVAO informed the Panel that the external stakeholder 
interviewees were selected by the organisations they represent and that the 
NVAO staff interviewees were selected according to the criteria provided by the 
Panel. All those interviewed had read the SER and most stakeholders had been 
given the opportunity to comment on a draft version of it. 

The SER contains valuable insights and statements. The Panel would, however, 
have appreciated to have been provided before the site visit with more factual, 
detailed evidence e.g. linking NVAO's standards and procedures more explicitly 
with the ESG. 

The SER, its attachments and added documentation constituted the frame of 
reference for the interviews during the site visit, which in turn provided further 
oral evidence related to the written documentation. 
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The Panel appreciates the fact that a number of relevant documents, originally in 
Dutch, were provided in an English translation and also notes that the 
assessment and accreditation reports (to be found on NVAO's web site) and 
several additional documents provided during the site visit were available in 
Dutch only. In order to assess the content of specific reports available only in 
Dutch, these reports were read and described to the Panel by the sole Panel 
member who understands the language. 

After the second day of the site visit, the Panel held an internal meeting where it 
agreed on the preliminary conclusions related to level of compliance of NVAO in 
relation to each of the standards in part 2 and 3 of the ESG. The secretary of the 
Panel then drafted the report in cooperation with the rest of the Panel. The draft 
report was submitted to NVAO for factual verification on August 27th, 2012 and 
with reference to ENQA standards NVAO was given two weeks to comment on 
the report. The final report was submitted to ENQA on September28, 2012. 

In relation to its conclusions, the Panel finds it important to note that it has 
assessed NVAO's level of compliance with the standards and also taken into 
account NVAO's practice in relation to some of the indicators listed in the 
guidelines. It has also provided a number of recommendations. It is the ambition 
of the Panel that this approach reflects the new policy for external reviews of 
agencies decided by the ENQA board which came into effect on July l st , 2011. 
The policy states (among other things) that the enhancement aspect of the 
reviews shall be strengthened in the second round and the agency thus be given 
more recommendations for further development than in the first round of 
reviews. 

The panel has decided to present its understanding of the main points of the 
standards and/or the guidelines in the subsections labeled "key elements" in 
chapter 4. 

vi. The national (and international) context of the review 

During the site visit, the interviewees mentioned several times that NVAO is in a 
"transition state" as models and procedures for quality assurance are currently 
being changed. It is the view of the Panel that a situation of change and 
development is to be expected for most quality assurance agencies. The Panel 
recognises that NVAO and its partners have work ahead in order to establish the 
institutional assessment approach in Flanders as well as to consolidate the new 
practice in the Netherlands. The Panel encourages all the parties to learn from 
the experience gathered thus far. 

*** 

Acknowledgement 

The Panel would like to thank formally all those that engaged in the process, 
including all stakeholders who were generous with their time, feedback and 
insights. The hospitable and open approach by the management and sta ff of 
NVAO was very much appreciated by the Panel. 
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4. Compliance with the ENQA membership criteria linked 
to the relevant European Standards and Guidelines 
(ESG) 

4.1 ENQA criterion 1 - Activities (ESG 3.1, 3.3) 

Agencies should undertake external quality assurance activities (at institutional 
or programme level) on a regular basis. The external quality assurance of 
agencies should take into account the presence and effectiveness of the external 
quality assurance processes described in Part 2 of the European Standards and 
Guidelines. The external quality assurance activities may involve evaluation, 
review, audit, assessment, accreditation or other similar activities and should be 
part of the core functions of the agency. 

Key elements 

Concerning the key elements "quality assurance activities", "a regular basis" and 
"core function" they are concrete and perceived in the same way by all. They are 
fully met by NVAO's accreditation through the 6/8 year cycle and the fact that 
NVAO either conducts the quality assurance exercise itself and makes the 
accreditation decision (institutional audits in the "new" system in the 
Netherlands, initial programme assessments in both NL and FL) or makes 
decisions on the basis of the quality assessments performed by quality assurance 
agencies (assessments of existing programmes in both NL and FL). 

The second paragraph: "... should take into account the presence and 
effectiveness of the external quality assurance processes described in Part 2 of 
the ESG" refers to ESG Part 2 (8 standards), which again refers to ESG Part 1 (7 
standards), each Standard having a variable number of guidelines. 

Evidence and analysis 

NVAO's compliance with these standards was considered as fully met in the 
ENQA review of 2007. The Panel relied upon the assessment made in this first 
ENQA review of NVAO in relation to general characteristics that have not 
changed. Where relevant, the Panel has also taken into account the ENQA 
reviews of QANU, VLHORA and VLIR. 

Reporting on the level of compliance by NVAO with the ESG Part 2 in the 
following section is structured in accordance with each of the standards 2.1 to 
2.8: sections 4.1.1 to 4.1.8 below contain these assessments. NVAO's level of 
compliance with ESG 3.1 is linked to compliance with ESG 2.1 - 2.8. 

Recommenda tions 

The recommendations of the Panel are presented at the end of each main section 
of chapter 4. The first set of recommendations is thus listed at the end of section 
4.1 
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4.1.1 ESG 2.1 Use of internal quality assurance procedures 

Standard 

External quality assurance procedures should take into account the effectiveness 
of the internal quality assurance processes described in Part 1 of the European 
Standards and Guidelines. 

Key elements 

"the effectiveness of internal quality assurance processes", "described in Part 1". 
Evidence 

Assessment of the HEI's internal quality assurance is a major focus in the Dutch 
framework for institutional audits (standards 3, 4 and 5). Internal quality 
assurance processes are also incorporated as a separate theme (theme 5) in 
both the Flemish (initial) accreditation frameworks and in the Dutch framework 
for extensive (initial) programme accreditation. 

Each of the quality aspects included in the standards contained in the Part 1 of 
the ESG are included in at least one of the frameworks for assessment. The 
frameworks are complementary to each other. 

NVAO is responsible for all processes in relation to the new institutional audits in 
the Netherlands and the assessment of new programmes in both the Netherlands 
and Flanders. In both the Netherlands and in Flanders, NVAO is also responsible 
for accreditation decisions on existing programmes as well as for specifying 
frameworks for assessment to be applied by the quality assurance agencies and 
their panels. These frameworks are formulated by NVAO in consultation with, 
among others, these quality assurance agencies. In the process of assessing the 
reports related to assessments of existing programmes, NVAO checks according 
to the SER and interviews during the site visit whether the agencies truly adhere 
to the frameworks. 

The conclusion reached in the reviews of QANU, VLHORA and VLIR are that these 
agencies fully comply with ESG 2.1 

Analysis 

Both the documentary and orally presented evidence are convincing. 

Besides the frameworks for assessments of which the ones in use in the 
Netherlands contain explicit references to the ESG, the Panel has been provided 
with tables showing the relationship between the focus in each of the different 
forms of assessments and the ESG Part(s) 1 (and 2). During the site visit, both 
the executive board of NVAO and staff members stressed that the new Dutch 
system for external quality assurance has been developed specifically to be 
aligned with the ESG. It was confirmed that institutional audit examines the 
quality assurance within each HEI as a whole. 

Indications for the effectiveness of the focus on internal quality assurance of 
HEIs is provided by the fact that representatives from the Dutch ministry 
expressed the view that one of the main achievements of the new system for 
external quality assurance in the Netherlands is that it contributes to the 
improvement of the internal quality assurance within institutions. The General 
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Board expressed the view that there is an increasing emphasis on development 
of a quality culture in all HEIs, and similarly the umbrella organisations of the 
HEIs in the Netherlands expressed the view that the institutional audits are 
useful in encouraging debate within institutions regarding the overall mission, 
strategie directions etc. 

With respect to Flanders, the Panel considers that the fact that VLIR and VLHORA 
both have been accepted as full ENQA members is sufficiënt proof that they pay 
due attention to Part 1 of the ESG in their own processes and thus provides 
further support to the conclusion reached by the Panel. 

Conclusion 

NVAO fully complies with ESG 2.1. 



4.1.2 ESG 2.2 Development of external quality assurance processes 

Standard 

The aims and objectives of quality assurance processes should be determined 
before the processes themselves are developed, by all those responsible 
(including higher education institutions) and should be published with a 
description of the procedures to be used. 

Key elements 

Developing aims and objectives of quality assurance processes involving 
stakeholders, "published with a description of the procedures to be used", no 
unnecessary interference with normal work (the latter is from the guidelines). 

Evidence 

According to the SER presented by NVAO and confirmed by those interviewed 
during the site visit, both the frameworks still in use in Flanders and the new 
frameworks for accreditation and institutional audit in use in the Netherlands 
were designed in consultation with representatives of institutions, quality 
assurance agencies and other assessment experts. Moreover, the new 
frameworks used in the Netherlands were designed with reference to a thorough 
evaluation of the previous system, followed by various forms of discussions with 
external stakeholders and a subsequent pilot-test among a number of higher 
education institutions and study programmes in both the Netherlands and 
Flanders. 

The frameworks were then discussed in parliament and after being decided upon 
they were published online on NVAO's website well before the start of the 
implementation of the systems in both countries. The frameworks include a 
description of the procedures to be used. The frameworks for the Netherlands 
also containa very short and general description of the aims and objectives of the 
processes, but a similar description is not part of the current Flemish frameworks 
The aims and objectives of the processes in both the Netherlands and Flanders 
are described indirectly, but not specifically, in NVAO's new Strategie Policy 
Statement, which has been published on NVAO's website after the Panel's visit. 

According to both the SER and those interviewed during the site visit, an explicit 
ambition of the new system, and something NVAO has committed itself to 
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achieve, is that it should lead to a 25% reduction of the administrative burden 
involved in programme assessments. The idea is that this reduction is to be 
achieved by the way limited programme assessment and IA complement each 
other. Process oriented aspects that programmes have in common or which are 
organised at a higher level, such as facilities and systems of internal quality 
assurance, are (NL) or will be (FL) assessed once in the institutional audits and 
not repeated at programme level anymore. 

Analysis 

The impression of the Panel is that all relevant stakeholders, including HEIs, have 
indeed been involved in the development of the assessment frameworks. As 
noted in the external review of NVAO of 2007, the evaluation of the level of 
compliance in relation to this Standard has to take into account the fact that 
accreditation in Flanders and The Netherlands is regulated by law. This implies 
that the global aims and objectives have been determined in a democratie 
process by legislation and that the frameworks have been approved by the 
respective ministers. In this process the impression from the SER and the 
interviews during the site visit is that the frameworks that were developed by 
NVAO do take into account comments made by the stakeholders. 

The fact that the frameworks do not contain explicit statements of the aims and 
objectives of the assessment processes surprises the Panel and is a shortcoming, 
but since the aims and objectives are presented elsewhere (albeit not very 
publicly) and since all relevant stakeholders have been involved in the design of 
the processes, the Panel does not find this to be a critical issue in relation to the 
overall assessment of NVAO's level of compliance with ESG 2.2 in itself. 
However, NVAO has an improvement potential in presenting aims and objectives 
in basic introductory parts to each of their frameworks: What kind of effects 
(control and/or enhancement) are aimed for connected to each accreditation 
procedure. Such an element would also facilitate the understanding of the design 
of the processes (section 4.1.4) 

In its SER, NVAO demonstrates that it is aware of the fact that it may take some 
time before the 25% reduction of administrative burden will be achieved as the 
implementation of the new frameworks implies that institutions and programmes 
have to get used to the new information files and self-evaluation reports. But by 
having set this ambition and committed itself to achieve it and play an active role 
in monitoring whether the aim will be achieved, the Panel is convinced that NVAO 
does what it can to ensure that the demands on institutions are no greater than 
what is necessary for the achievement of the objectives of the new system. 

Conclusion 

NVAO substantially complies with ESG 2.2. 



4.1.3 ESG 2.3 Criteria for decisions 

Standard 

Any formal decisions made as a result of an external quality assurance activity 
should be based on explicit published criteria that are applied consistently. 
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Key elements 

"formal decisions", "explicit published criteria", "applied consistently". 

By "formal decisions" the Panel understands this to refer to the accreditation 
decisions that are executed by NVAO on the basis of their own assessments 
(institutional audits in the Netherlands and initial programme assessments in the 
Netherlands and Flanders) and assessment reports produced by quality 
assurance agencies (assessments of existing programmes in the Netherlands and 
Flanders). 

"Explicit published criteria" are understood as 

1. criteria that underpin quality as the backbone of the assessments 
performed by the experts/peers 

2. in the context of decision-making: criteria/procedural elements regulating 
the appointment and work of experts and also explaining the process from 
the presentation of an assessment report to the subsequent accreditation 
decision 

3. criteria for judging consistency containing indicators for deciding on any 
threshold values in use and their consequences. 

Evidence 

The frameworks contain standards for the different types of quality assessments 
and accreditations. 

The frameworks contain descriptions on the composition of Panels and on 
assessment processes. Regarding the use of threshold values ("scores"), the 
frameworks instruct the use of three or four levels dependent on the activity 
being undertaken. Chapter 8 in the assessment frameworks in the Netherlands 
gives examples illustrating the use of the assessment scales for programme 
assessments. Chapter 9 describes how the judgments of the Panel in relation to 
individual standards must affect the overall judgment of the programme in case 
of programme assessments and the overall judgment of the institution in case of 
an institutional audit. 

Whereas the framework for the institutional audits in the Netherlands contains a 
description of the consequences of the different possible outcomes (a positive, a 
conditional positive and a negative judgment) of an audit (section 2.5), the 
corresponding sections in the frameworks for (initial) programme assessments in 
the Netherlands only present the different possible conclusions that NVAO can 
reach. 

In its SER, NVAO states that it checks the quality of the assessment procedure 
and the quality of the programme by means of an analysis of the assessment 
report. If the report raises questions, NVAO requests additional information from 
the panels. Finally NVAO (the Executive Board) makes the accreditation decision. 
Concerning the process starting from the submission of an assessment report to 
an accreditation decision, the assessment frameworks for Flanders contain a 
description of the rules governing accreditation decisions (FL section 2.4.2). The 
Panel did not find evidence of a similar procedural description in the Dutch 
frameworks. 
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The SER states that NVAO has put in place several measures to ensure the 
consistency of the evaluation of assessment reports produced by panels, 
including: 

■ The use of detailed manuals for the evaluation of reports, which are 
updated on a regular basis. 

■ Frequent staff meetings where issues regarding the evaluation of 
assessment reports are discussed. 

■ Each application is evaluated by a policy advisor and a board member. If 
the evaluation necessitates this, the NVAO asks other staff members or 
board members for a second opinion. A final decision on an application is 
taken by all board members in a joint session on the basis of a detailed 
recommendation. 

The representatives of the General Board of NVAO interviewed during the site 
visit further noted that the General Board looks at consistency of decisions made 
by the Executive Board. 

Analysis 

When NVAO underwent a similar review in 2007, the quality assurance activities 
and criteria were alike for the Netherlands and Flanders. Today, NVAO states 
that the regulations are still the same in Flanders as it was then. In 2007 the 
assessments at this point concluded with "fully compliant". This Panel relies upon 
the conclusion made in 2007 concerning the Flemish situation since no change 
has been made yet. In the present assessment the Panel focused on the new 
Dutch system that was introduced in 2011. 

Whereas the assessment frameworks for Flanders contain precise descriptions of 
the rules that NVAO's formal accreditation decisions are based on, the 
assessment frameworks for the Netherlands do not contain similar descriptions. 
Since the assessments of existing programmes are executed by the Dutch quality 
assurance agencies and thus not by NVAO, the Panel finds it critical that it is not 
made clear how NVAO reaches its accreditation decisions. In other words, it was 
unclear to the Panel which criteria NVAO uses, if any, for deciding whether to 
follow or deviate from the conclusions reached by a quality assessment panel 
and how NVAO assures consistency in this respect. NVAO did not clarify this 
issue during discussions with the Panel during the site visit. It is important that 
NVAO respects the assessments made by the panel experts and that NVAO 
makes its considerations and ultimate accreditation decision in a manner and 
based on clear evidence that is completely transparent in all cases to all the 
involved parties. 

In practice, however, the internal procedures/the mechanisms within NVAO 
appear to promote consistency in the decision making process and the site visit 
revealed no strong feelings on this matter. 

Nevertheless, the Panel noted, during the site visit, a view that there is still a 
possibility for improvement regarding the consistency of accreditation judgments 
reached. NVAO also appears to be aware of the challenges regarding consistency 
as "the points of attention" raised in relation to ESG 2.3 in its SER are both 
concerned with the issue of consistent application of criteria. Furthermore, one of 
the 6 issues NVAO list as those it should pay attention to in the next years in 
order to improve its operation and the operation of the accreditation system in 
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the Netherlands and Flanders is: The large number of applications necessitates 
permanent attention to consistency in decision making, where criteria and 
mechanisms are closely connected. 

Conclusion 

IWAO partially complies with ESG 2.3. 



4.1.4 ESG 2.4 Processes fit for purpose 

Standard 

All external quality assurance processes should be designed specifically to ensure 
their fitness to achieve the aims and objectives set for them. 

Key elements 

This Standard is in itself wide and abstract and therefore may be subject to many 
different interpretations. However, it is followed by quite extensive guidelines. In 
order to establish a common understanding of quality related to this Standard, 
the Panel has included the guidelines as a starting point of its analysis: 

"Quality assurance agencies within the EHEA undertake different external 
processes for different purposes and in different ways. It is of the first 
importance that agencies should operate procedures which are fit for their own 
defined and published purposes. Experience has shown, however, that there are 
some widely-used elements of external review processes which not only help to 
ensure their validity, reliability and usefulness, but also provide a basis for the 
European dimension to quality assurance. 

Amongst these elements the following are particularly noteworthy: 

• insistence that the experts undertaking the external quality assurance 
activity have appropriate skills and are competent to perform their task; 

• the exercise of care in the selection of experts; 

• the provision of appropriate briefing or training for experts; 

• the use of international experts; 

• participation of students; 

• ensuring that the review procedures used are sufficiënt to provide 
adequate evidence to support the findings and conclusions reached; 

• the use of the self-evaluation/site visit/draft report/published 
report/follow-up model of review; 

• recognition of the importance of institutional improvement and 
enhancement policies as a fundamental element in the assurance of 
quality." 

In addition to the bullet points above, the Panel is concerned with the length of 
time that elapses from presenting the assessment report until an accreditation 
decision has been made. When time is unpredictable and/or running too long, it 
is a challenge for the involved parties to keep up an interest in the result and a 
motivation for action upon it. This is especially essential to support the aim of 
quality enhancement. 

Evidence 

The following subsections focus exclusively on the presence and characteristics of 
the elements of the processes that are listed in the guidelines to ESG 2.4. 
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Characteristics of the panels 

The panels' required competencies are described for all activities in the various 
assessment frameworks. 

NVAO appoints the panels for institutional audits (NL) and for initial programme 
assessment in both the Netherlands and Flanders. In terms of the assessment of 
existing programmes in the Netherlands, the panels are nominated by the HEIs, 
and the final composition is determined by the quality assurance agencies. Final 
approval of the composition of the panels is made by NVAO. 

According to the representatives of the Dutch quality assurance agencies 
interviewed during the site visit, approximately 25% of the suggested panel 
members are queried by NVAO. In Flanders panels for the assessment of existing 
programmes are decided by the Flemish quality assurance agencies and 
approved by the Recognition committee, an independent external body that 
checks the independence of proposed members of review panels, before the 
assessment takes place. 

All of the Dutch assessments frameworks include a section describing how the 
independence (both actual and perceived) of the panel members is secured and 
the requirements that the panels must meet. In all cases requirements with 
regards to the skills and competences of the panel members are specified. For all 
types of assessment in the Netherlands it is also (among other things) a 
requirement that the panels include a student and a member with international 
expertise. Requirements concerning the composition of panels are described in 
the Flemish accreditation decree and in the NVAO rules about quality assurance 
agencies and both VLIR and VLHORA have requirements about student 
participation and international expertise in their panels. 

Training of panel members 

Student members of the panels for initial programme assessments (both NL and 
FL) and potential chairs for the institutional audits in the Netherlands are trained 
by NVAO. The training for institutional audit lasts two days and includes, among 
other things, role play. Panel members participating in the assessments of 
existing programmes in the Netherlands and Flanders are instructed or trained 
by the quality assurance agencies, but NVAO offers a "Train the trainer module" 
for programme assessment. 

For all types of the assessment panels in the Netherlands, those secretaries who 
are not panel members are trained by NVAO. When the site visit of NVAO took 
place approximately 200 secretaries had been trained. The training focuses on 
the assessment frameworks, how to use them, how to guide the panel members 
etc. Furthermore, secretaries participate in two reflection days per year at NVAO. 

Provision of adequate evidence 

The frameworks for assessments describe the review procedures in detail and 
include sections stressing the importance of providing adequate evidence to 
support the findings and conclusions reached. 
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Whether findings and conclusions are supported by adequate evidence is also the 
main focus when NVAO assesses the reports produced by the quality assurance 
agencies - and the main reason for sometimes asking for more information. 

The use of self-evaluation/site visit/draft report/published report/follow-up model 
of review 

Self-evaluation, site visit, draft report and published report all form parts of all 
the assessment processes in the Netherlands and Flanders. Follow-up is part of a 
process in the assessment of existing programmes if a recovery period has been 
granted. For initial programme assessments and for institutional audits, a follow- 
up review takes place only where positive decisions are made subject to 
conditions. 

Recognition of the importance of institutional improvement and enhancement 
policies as a fundamental element in the assurance of quality 
According to the SER, the new accreditation system in the the Netherlands 
heralds a new approach. The SER describes the new system as challenging the 
institutions to set higher targets for their programmes and their quality work in 
general and thus improving quality. Furthermore, it is stated in the SER that the 
new frameworks encourage the development of quality assurance and of a 
quality culture within the entire institution and that the new system allows for a 
more thorough discussion about systematic quality assurance and quality culture 
at institutional level and the quality of content and learning in individual 
programmes. Staff interviewed noted that reports are directed towards 
curriculum development and improvement. 

Time from submission of report until decision has been made 
Interviews revealed that the time from presentation of assessment report until 
decision has been made by NVAO is of varying length and, in some cases, 
exceeds one year. The interviewees not employed at NVAO did not know why the 
delay is so unpredictable. Some also expressed the opinion that, in some cases, 
conditions within the HEI had changed substantially between the time of the site 
visit and the receipt of the review report and that the decision and 
recommendations were no longer truly relevant. The interview with staff 
revealed that NVAO is challenged by the fact that most of the HEI's apply for 
accreditation at the same time of the year which means that there is a great 
imbalance over the year in the amount of applications that NVAO has to handle. 

Analysis 

The impression of the Panel from reading the SER and the assessment 
frameworks is that most processes are fit for purpose. This impression is further 
strengthened by the fact that none of those interviewed during the site visit 
questioned or criticised the processes, except for some overly long delays in 
delivery of the final report. 

As the description above reveals, the processes include all elements listed in the 
guidelines to ESG 2.4. Interviews during the site visit gave the impression that 
the panels are considered to be competent and doing a good job. In the 
interview with the Dutch quality assurance agencies, the view was raised that for 
those secretaries who have already been trained by a quality assurance agency, 
the NVAO training may be feit as a waste of time; the quality of the training 
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provided by NVAO was also questioned, but this view was not raised in any other 
interviews. 

Furthermore, the Panel has noted that in the reviews of VLIR and VLHORA the 
composition of the panels for the assessment of the Flemish programmes and 
the training of these panel members, as well as the assessment processes in 
Flanders in general, have been positively assessed and approved. 
The issue of the delay to decision-making is not specifically addressed in ESG. 
The Panel considers this issue as being a part of its considerations of "fit for 
purpose", which is illustrated by the information during interviews as given 
above. There is, however, no reason to believe that there are too many cases of 
unpredictable and inappropriate long timelines, but in the view of the panel the 
delays are still a critical issue. 

Generally, the impression of the Panel is that the new system in the Netherlands 
and not least the inclusion of institutional audits and the possibility for granting a 
recovery period do indeed promote more focus on quality improvement and 
enhancement than the previous system did. This impression is supported by the 
SER which, based on an evaluation of the first institutional audits, concludes that 
the assessed institutions consider the audit a valuable learning experience that 
will positively affect the quality of the programmes they offer and causes staff to 
reflect more intensively on internal quality assurance at an institutional level. At 
the same time the Panel finds that the lack of focus on recommendations for 
improvement in the decision reports in case of positively accredited programmes 
in the Netherlands suggests that the focus on quality improvement and 
enhancement can still be further strengthened. 

Conclusion 

NVAO substantially complies with ESG 2.4. 



4.1.5 ESG 2.5 Reporting 

Standard 

Reports should be published and should be written in a style which is clear and 
readily accessible to its intended readership. Any decisions, commendations or 
recommendations contained in reports should be easy for a reader to find. 

Key elements 

"Intended readership", "readily accessible" 
Evidence 

As regards assessments of existing programmes, in Flanders, the institution 
applies for accreditation of a programme with NVAO, submitting the assessment 
report of the panel produced and published by the evaluating agency. In the 
Netherlands, the assessment report is published after completion of the 
accreditation procedure. 

In both cases the reports have a summary, which NVAO can use to provide 
information to the public. NVAO requires the summary to be very concise in 
order to inform interested readers at a glance about the most relevant 
characteristics of the programme. In addition, every assessment report contains 
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a scoring table and a separate paragraph with recommendations. The staff 
interviewed during site visit stated that NVAO is the primary intended readership 
for the programme assessment reports, HEIs the second one and the general 
public the third one. The staff also stated that summaries have been introduced 
primarily for the benefit of students and external stakeholders. 

The NVAO accreditation decision reports are based on the assessment reports 
and contain a summary of the findings and discussions contained in these reports 
(the summary and the scoring table are integral part of the decision document), 
followed by the formal decision of NVAO. 

For all types of assessments, the reports and decision documents have a fixed 
and structured format with the purpose of increasing readability. In 2012, NVAO 
started an evaluation project to investigate how well the new reports respond to 
the needs of the intended readers. 

Both NVAO's decisions and the panel assessment reports are made public on 
NVAO's website ( www.nvao.net ). The site is equipped with a search tooi to find 
information about a specific programme or an institution. NVAO's website lists all 
programmes in the Netherlands and Flanders which were submitted for 
accreditation. 

Analysis 

The reports produced and published by the Flemish quality assurance agencies 
have generally been positively assessed in the external reviews of VLHORA and 
VLIR. In the case of VLHORA, some criticism was raised (in 2008) about the 
clarity with regard to the intended readership and the lack of a summary which 
led to an assessment of VLHORA as being "only" substantially compliant; a 
recommendation to include a summary in the report was made. Since the 
Flemish reports - like the new Dutch reports - now have to include a summary, 
but have not otherwise been affected by the introduction of the new system in 
the Netherlands, the Panel rests its assessment of the Flemish reports on the 
external reviews of VLIR and VLHORA, leaving aside only the issue regarding the 
summary. 

Regarding the reports related to Dutch programmes, NVAO states in its SER that, 
with the introduction of the new system, NVAO agreed with the quality assurance 
agencies to improve the readability of the programme assessment reports. NVAO 
feels that the first results in the Netherlands are promising. NVAO expects that 
the main findings and conclusions of a report will be easier to read for students, 
employers and other involved groups. 

More specifically, NVAO finds, according to its SER, that, as a result of the 
introduction of limited programme assessment and extended programme 
assessment, the first panel reports produced in the new format are more 
informative and transparent than previous reports. 

During the site visit, the relevant external stakeholders were questioned about 
their view on the quality of the new Dutch reports. Both the representatives of 
the Dutch Ministry of Education, the Dutch umbrella organisations, the Dutch 
student organisation and the Advisory Board expressed a generally positive 
appreciation of the first reports and that the reports are good and have been 
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improved with the new system, but that there is still room for improvement (or a 
complete implementation of the new guidelines). The summaries were 
highlighted as a new and very positive element in the reports, not least in terms 
of making the reports more accessible to a wider audience including students. 
The main critique raised by the representatives mentioned above was that the 
intended readership of the reports is too narrow and (as a consequence) the 
reports are too technical to be easily read by the general public. 

In the course of the review, the Panel has looked into and assessed a number of 
the reports that have been published after the introduction of the new system in 
The Netherlands. The conclusion in relation to the reports following the new 
institutional audits as well as the ones for the revised form of (initial) limited and 
extended assessments is that they faithfully follow the relevant assessment 
framework of NVAO and that the structure is more or less identical across 
assessments within the same framework. All include a summary and are 
structured to cover description, analysis (including relevant evidence), 
conclusions and some forms of recommendations. They also contain sufficiënt 
preliminary explanation to enable a lay reader to understand the purposes of the 
review, its form, and the criteria used in reaching conclusions. The criteria used 
in making decisions are not explained in the NVAO accreditation decision reports 
(Cf. section 4.1.3 for an assessment of this). The writing style is literary with a 
few other steps to enlightening major issues or main findings. 

As for the accessibility the Panel finds that, despite the fact that the search 
facility of NVAO's web site makes it rather hard to find a specific assessment 
report, the assessment reports contain what NVAO needs to be able to reach an 
accreditation decision and thus fit the primary intended readership of the reports 
as defined by NVAO. Despite the introduction of a summary, the Panel is, 
however, not convinced about the general readability for other stakeholders and 
agrees with the external stakeholders who during the site visit characterised the 
reports as being too technical - and maybe also too verbose - to read for the 
general public, including (prospective) students. Thus the panel also agrees with 
the stakeholders that the intended readership of the reports defined by NVAO is 
too narrow. 

Having said this, The Panel has positively noted that NVAO has recently started 
an evaluation project to investigate how well the new reports respond to the 
needs of the intended readers besides NVAO itself. 

Conclusion 

NVAO substantially complies with ESG 2.5. 



4.1.6 ESG 2.6 Follow-up procedures 

Standard 

Quality assurance processes which contain recommendations for action or which 
require a subsequent action plan, should have a predetermined follow-up 
procedure which is implemented consistently. 

Key elements 

"recommendations for action", "a predetermined follow-up procedure" 
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"The objective is to ensure that areas identified for improvement are dealt with 
speedily and that further enhancement is encouraged." (Guideline) 

Evidence 

The programme assessment reports contain a scoring table and a chapter with 
recommendations. The institutional audit reports also contain a chapter with 
possible improvements/recommendations listed. When a conditional accreditation 
decision is reached in relation to an institutional audit or an initial programme 
assessment, a subsequent action plan is required. In case of the application of a 
recovery period in relation to an assessment of an existing programme, the 
institution is required to present a convincing recovery plan before the recovery 
period is granted. 

NVAO states, in its SER, that a formal follow-up procedure is not included as an 
overall obligatory part of the accreditation system. 

At the same time, the SER makes clear that a recovery period has been part of 
the programme accreditation system in Flanders since 2005 and is part of the 
new procedure for programme accreditation in the Netherlands. More 
importantly, it is stated that in the case of the application of a recovery period, 
the institution needs to present a convincing recovery plan before the recovery 
period is granted and that, at the end of the recovery period, the achieved 
improvements have to be assessed positively by an assessment panel before 
accreditation can be granted. 

Similarly, NVAO now has the authority to take a conditionally positive decision in 
the procedures of initial programme accreditation and institutional audit. In the 
case of a conditional decision in the Netherlands, a NVAO panel must follow-up 
and assess whether the programme or the institution meets the conditions within 
the set conditional period. 

In the case of an unconditionally positive decision, there is no follow-up about 
the extent to which the institution and/or the programme management 
implement the recommendations of the assessment panel. In these cases, the 
follow-up on these recommendations will be assessed in the subsequent 
accreditation procedure (after 6 to 8 years). 

In the 6 to 8 year period between being subject to an accreditation procedure, 
the Panel learned from the site visit that more and more programmes and 
institutions voluntarily engage in some forms of other reviews and publish 
reports based on these. 

Analysis 

In its SER, NVAO states: "An official follow-up procedure, as meant here in the 
ESG guideline, is not a part of the accreditation system either, hut two measures 
introducé elements that provide equivalent functionality to a follow-up procedure 



The panel finds that follow-up procedures have been strengthened in the 
Netherlands through the introduction of a recovery period similar to what has 
been in place in Flanders since 2005. 



23 



It is now only in the case of an unconditionally positive decision that a follow-up 
procedure is not applied. As ESG 2.6 states that quality assurance processes 
which contain recommendations for action or which require a subsequent action 
plan should have a predetermined follow-up procedure which is implemented 
consistently, and as unconditionally positive accreditation decisions do not 
contain recommendations or require a subsequent action plan the standards does 
not apply to this part of the quality assurance processes of NVAO. Therefore the 
view of the Panel is that NVAO's level of compliance with ESG 2.6 is adequate. 
Moreover the Panel is convinced that the objective of ESG 2.6, as expressed in 
the guidelines to the Standard, "to ensure that areas identified for improvement 
are dealt with speedily and that further enhancement is encouraged" is fulfilled. 

Conclusion 

NVAO fully complies with ESG 2.6. 



4.1.7 ESG 2.7 Periodic reviews 

Standard 

External quality assurance of institutions and/or programmes should be 
undertaken on a cyclical basis. The length of the cycle and the review procedures 
to be used should be clearly defined and published in advance. 

Key elements 

"Cyclical basis", "review procedures", "clearly defined", "published in advance" 
From the guidelines: "demands on institutions should not be greater than are 
necessary" 

Evidence 

The accreditation period of programmes in the Netherlands is six years, in 
Flanders it is eight years. After an initial accreditation, reassessment of the 
programme and reaccreditation should be finalized within six years (in the 
Netherlands) and within the length of the programme and two years (in 
Flanders). These cycles are mandated by Dutch and Flemish legislation. 

The Dutch HEI's are subject to an institutional audit every six years. 

Analysis 

The description above clearly reveals that external quality assurance in both the 
Netherlands and Flanders are undertaken on a cyclical basis and that the length 
of the cycle and the review procedures are clearly defined and published in 
advance. 

Conclusion 

NVAO fully complies with ESG 2.7. 
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4.1.8 ESG 2.8 System-wide analyses 

Standard 

Quality assurance agencies should produce from time to time summary reports 
describing and analysing the general findings of their reviews, evaluations, 
assessments etc. 

Key elements 

"from time to time", "summary reports", "describing and analysing general 
findings" 

Evidence 

In the previous review in 2007, NVAO was considered only partially compliant 
with ESG 2.8 and was advised to give more attention to the production of 
system-wide and comparative analyses. 

In the past years, NVAO completed a number of comparative analyses within 
clusters or domains. The analyses are mentioned in the SER and are listed and 
described in the attachments to it. An evaluation of the former accreditation 
system and of the pilots undertaken in order to develop the new system are 
other examples of system-wide analyses mentioned in the SER, but these have 
not been included in the list in the attachments and at the time of the review it 
was therefore unclear to the Panel whether they resulted in any form of reports 
or what kind of results they produced. 

In relation to the future, NVAO's new strategie policy underlines the importance 
of comparative and system-wide analyses and, according to the SER, NVAO 
recruited a new staff member assigned to do quantitative and qualitative 
research to underpin the analyses and a number of different system-wide 
analyses have been planned. 

Analysis 

From reading the description and analysis of the last external review of NVAO 
which led to the conclusion that NVAO only partially complied with ESG 2.8 as 
well as the present SER and list of comparative analysis published by NVAO since 
2007, the Panel finds that NVAO has substantially improved its practice in 
relation to ESG 2.8. 

The Panel notes, however, that NVAO itself is relatively critical about its 
achievement so far. In its SER, it states that despite its substantial effort 
regarding comparative analyses, it did not have a clear policy on this issue in 
recent years. It is further stated that in the new Strategie Policy Statement 
(2012 - 2016) the NVAO task of system-wide and comparative analyses is 
emphasized and should result in a clear agenda on this topic for the coming 
years and "that NVAO has to work on implementing its new policy regarding 
system-wide and comparative analyses". Finally it concludes that one of its 
points of attention is to implement a clear policy and activities regarding system- 
wide analyses through yearly planning. 

Some of the stakeholder representatives interviewed during the site visit 
expressed the view that they would like NVAO to do more system-wide analysis. 
Issues such as internationalisation, student experience and comparative analysis 



25 



of how the quality assurance agencies in Flanders and the Netherlands work were 
mentioned in this regard. This may require NVAO to adopt a more cluster-based 
approach than is currently the case. 

As the Panel is informed, the situation is different in Flanders and in the 
Netherlands: the Flemish agencies operate on clusters of similar programmes 
within all HEIs which the Panel believes facilitates system-wide comparisons, 
while in the Netherlands - despite cluster assessment of university programmes 
since 2003 the process is geared more towards assessments of single institutions 
or single programmes. 

In its assessment of NVAO's level of compliance with ESG 2.8, the Panel has 
given the critical self-assessment contained in the SER of NVAO a high weighting. 

Conclusion 

NVAO substantially complies with ESG 2.8. 



4.1.9 ESG 3.1 Use of external quality assurance procedures for higher 
education 

Standard 

The external quality assurance agencies should take into account the presence 
and effectiveness of the external quality assurance processes described in Part 2 
of the European Standards and Guidelines. 

Key elements 

This Standard is understood holistic in its full text, there are no key words as 
such in this formulation. 

Evidence and analysis 

In section 4.1.1 to 4.1.8 the Panel has assessed the NVAO's level of compliance 
with ESG 2.1 to 2.8 and thus with Part 2 of the European Standards and 
Guidelines. The outcome is that NVAO is considered fully compliant with ESG 2.1, 
2,6 and 2.7, substantially compliant with ESG 2.2, 2.4, 2.5 and 2.8 and partially 
compliant with ESG 2.3. 

Conclusion 

Based on its assessment of NVAOs level of compliance with ESG 2.1 to 2.8, the 
Panel concludes that NVAO substantially complies with ESG 3.1 



4.1.10 ESG 3.3 Activities 

Standard 

Agencies should undertake external quality assurance activities (at institutional 
or programme level) on a regular basis. 

Key elements 

This Standard is understood as holistic in its full text, there are no key words as 
such in this formulation. 
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Evidence 

The tasks of NVAO are described in the Treaty. They are further stipulated in the 
Netherlands in the Dutch Act on higher education and research and can be 
summarized as the (initial) accreditation of programmes of institutions of higher 
education and the assessment of these institutions (Institutional audit). 
In Flanders, the Flemish Act of 4 April 2003 forms the legal basis for (initial) 
accreditation in higher education and stipulates that the responsibility for (initial) 
accreditation of programmes is assigned to NVAO. Since it was established in 
2003, NVAO has processed nearly 5000 applications for accreditation. 

NVAO can be requested to carry out additional tasks by the Ministers of higher 
education in both countries if these assignments support or supplement NVAO's 
primary assignment. 

Analysis 

The Panel finds that the description above as well as sections 4.1.1 to 4.1.8 
clearly illustrates that NVAO undertakes external quality assurance activities on a 
regular basis. 

Conclusion 

NVAO fully complies with ESG 3.3. 



Conclusion ENQA criterion 1 

Based on its assessment of NVAOs level of compliance with 3.1 (including ESG 
2.1 to 2.8) and 3.3, the Panel concludes that NVAO substantially complies with 
the ENQA membership criterion 1/ESG 3.1 and 3.3 

R ecomm en da tions 

With reference to the evidence and analysis provided in section 4.1.1 to 4.1.10 
the Panel recommends that NVAO: 

• makes the link between the NVAO standards and ESG Part 1 more explicit in 
the assessment frameworks; 

• refines the descriptions of the aims and objectives, ensures that they are 
prominent in the frameworks and show how the various elements of the 
frameworks contribute to the aims and objectives; 

• formulates explicit and public criteria about how it reaches its accreditation 
decisions in relation to the programme assessments in all cases; 

• establishes a clear procedure on how to handle cases where the conclusions 
in the assessment report are not accepted by NVAO; 

• strengthens the predictability of the timeframe and efficiency of its decision- 
making process; 

• further strengthens the focus on quality improvement and enhancement of 
the HEI's; 

• clarifies a) the purpose of every kind of report, b) the readership and c) the 
needs of the various kinds of readers in order to enhance the readability; 

• gives a high priority to: a) the identification of the interested parties for 
system-wide analyses and of their needs; b) the definition of a realistic 
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schedule of system-wide analyses; and c) the production of system-wide 
analyses corresponding to the needs which were identified. 
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4.2 



ENQA criterion 2 / ESG 3.2: Official status 



Standard 

Agencies should be formally recognised by competent public authorities in the 
European Higher Education Area as agencies with responsibilities for external 
quality assurance and should have an established legal basis. They should 
comply with any requirements of the legislative jurisdictions within which they 
operate. 

Key elements 

This Standard is understood holistically in its full text, there are no key words as 
such in this formulation. 

Evidence 

NVAO is the official, public and bi-national accreditation organization in higher 
education in the Netherlands and Flanders. lts structure and duties are described 
in a bi-national Treaty (2003) and in both national legislations. NVAO reports to 
the Dutch and Flemish Parliaments via their Ministers of (Higher) Education. 
NVAO's annual report is used for accountability purposes. 

Analysis 

From the description in the SER and the legal documents it refers to it is evident 
to the Panel that NVAO has an established legal basis and is formally recognised 
as required by ESG 3.2 

Although the Panel did not explicitly ask about it during the site visit, the fact 
that no one raised any points which could suggest that NVAO does not comply 
with requirements of the legislative jurisdictions within which it operates leads 
the Panel to conclude that NVAO fully complies in this respect. The fact that all 
external stakeholders expressed a general satisfaction with the work of NVAO 
and that NVAO was considered fully compliant with ESG 3.2 in the 2007 review, 
supports this assessment. 

Conclusion 

NVAO fully complies with ENQA criterion 2 / ESG 3.2. 
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4.3 



ENQA criterion 3 / ESG 3.4: Resources 



Standard 

Agencies should have adequate and proportional resources, both human and 
financial, to enable them to organise and run their external quality assurance 
process(es) in an effective and efficiënt manner, with appropriate provision for 
the development of their processes, procedures and staff. 

Key elements 

This Standard is understood holistically in its full text, there are no explicit key 
words as such in this formulation. However, the operationalization of the 
Standard: that is, what evidence should be presented and how it should be 
analysed by the Review Panel, is more of a question. 

Evidence 

NVAO has an annual budget of approximately € 6 million, which is financed 
jointly by the Netherlands (60%) and Flanders (40%). Each year, NVAO draws 
up a budget which is then decided upon by the Committee of Ministers and both 
Parliaments. In the Netherlands and Flanders, NVAO applies fixed rates by law. 
The rate for an application for initial accreditation is maximum € 15,000 in the 
Netherlands and € 5,000 in Flanders. For an application for accreditation the rate 
is € 750 in the Netherlands and € 500 in Flanders. In the case of additional tasks 
and assessment assignments abroad, NVAO charges cost-covering fees. NVAO 
has agreed with both ministries in the Netherlands and Flanders that additional 
tasks assigned by the ministries will be organized and budgeted on a project 
basis. NVAO informs the ministries in advance about the expected costs. The 
latter can decide whether to continue the project initiative or not. 

Since 2005, NVAO is located in the Parkstraat in The Hague, near the city and 
the Dutch parliament and the Dutch ministry of education. Several meeting 
rooms for smaller and larger groups are available for the organisation of 
conferences, seminars and meetings. Presentation and IT-facilities are up-to- 
date. In November 2011, a new information system was introduced to optimize 
the work-flow. 

The workforce of NVAO consists of 56 people (53FTEs): four executive Board 
members, one managing director, 30 policy advisors, one Dutch and one Flemish 
legal advisor (together 29FTEs) and 20 other (partly supporting) staff (19 FTEs). 
Four policy advisors are available for international assignments. The staff 
composition includes policy, legal and communication advisors and supporting 
staff (policy secretariat, finances and personnel, records department and general 
services). 

All academie staff hold a master's degree, with three members holding a PhD 
degree. The acquired professional experience of NVAO's staff is varied. All board 
members and the director have a longer career in (higher) education in board or 
management positions. Most staff acquired substantial work experience in higher 
education as teacher, developer, manager, researcher, inspector for higher 
education, or policy advisor before being employed by NVAO. A small group of 
junior staff was recruited to fulfill a kind of traineeship before being fully 
employable in NVAO's work processes. NVAO has defined core competencies of 
its staff. 
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Each member of staff pa rtici patés in a performance review once a year. On this 
occasion staff have the opportunity to ask for further training if they find it 
relevant. 

Analysis 

According to its SER, NVAO has been funded sufficiently by both governments for 
its primary tasks in recent years and has received appropriate additional funding 
for the additional tasks that were assigned to the organization. Despite the fact 
that NVAO, like all public organizations, is currently subject to budget cuts, NVAO 
states that these cuts have not affected its operations up to now. As the critical 
issue related to respecting deadlines does not appear to arise (primarily) from a 
lack of resources (cf. section 4.1.3) the impression of the Panel from the 
interviews during the site visit matches the self-assessment of NVAO. 

In its SER, NVAO concludes that good facilities are available at its location and, 
having visited and used these, the Panel fully agrees with that. 

The SER states that, over the years, NVAO has developed a good quality of staff 
with varied competences and work experiences. With reference to a recent 
benchmark study of staff development among five members of ENQA, including 
NVAO, it concludes that compared to similar members within the ENQA network, 
NVAO staff is on par. The positive assessment of the competences of staff was 
not questioned by any of those interviewed during the site visit. On the contrary, 
the representatives of the General Board, for example, expressed the belief that 
the quality of the staff of NVAO is one of the primary reasons for the credibility 
and respect with which NVAO is held in the community. 

The impression from the Panel's interview with NVAO staff is that they are 
satisfied with the possibilities offered for further training or other forms of 
competence development activities. On the job training and a mentoring system 
were mentioned by staff as some of the important forms of activities, particularly 
for newer members of staff. 

NVAO finds, in its SER, that further development of expertise of staff on specific 
themes in higher education should be improved and thus that it should invest in 
further development of specific, theme-oriented, higher education expertise of its 
staff. Although it may be a relevant priority for NVAO, the Panel does not in any 
way find that it is needed for the purpose of being fully compliant with ESG 3.4. 

Conclusion 

NVAO fully complies with ENQA criterion 3 / ESG 3.4. 
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4.4 ENQA criterion 4 / ESG 3.5: Mission statement 



Standard 

Agencies should have clear and explicit goals and objectives for their work, 
contained in a publicly available statement. 

Key elements 

"clear and explicit", "publicly available" and from the Guidelines: "the division of 
labour with relevant stakeholders in higher education" "documentation to 
demonstrate how the statements are translated into a clear policy and a 
management plan" 

Evidence 

In the "Strategie Policy Statement NVAO 2012 - 2016", NVAO has defined its 
mission as follows: 

NVAO is the independent and authoritative accreditation organisation set up by 
the Flemish and Dutch governments, whose primary goal it is to provide an 
expert and objective judgement of the quality of higher education in Flanders 
and the Netherlands. NVAO does this with a constructive, critical attitude, 
respecting the autonomy of institutions and their primary responsibility for the 
quality of their education, and with an open eye for the growing international 
context. NVAO is open, clear and transparent towards society and all concerned, 
especially the institutions of higher education and the students. 

At the time of the review the new Strategie Policy Statement was not published 
on the website of NVAO, but the mission statement was presented on the 
website. 

The new Strategie Policy Statement includes a section describing the context of 
the work assigned to NVAO; the division of labour with relevant stakeholders in 
higher education is described. NVAO states that it considers institutions to be 
primarily responsible for quality assurance and quality improvement. NVAO is 
required to assure that programmes meet the required standards and to 
stimulate the quality debate, giving account of its procedures, disseminating 
'good practices' and visiting institutions and their programmes. 

In relation to this the mission statement lists the three main tasks of NVAO as 
being: 

1. Assessing and assuring the quality of Dutch and Flemish higher education. 

2. Promoting the quality of higher education by promoting a culture of 
quality, aimed at regular assessment and continuous quality increase. 

3. Putting Dutch and Flemish sectors of higher education (institutions, 
programmes) on the map and strengthening their position by means of 
international cooperation. 

The Strategie Policy Statement also includes sections that translate the mission 
(and values and positioning) into a number of strategie goals and a strategie 
agenda for NVAO for 2012 to 2016. 
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Analysis 

It is evident from the description above that NVAO has clear and explicit goals 
and objectives for its work. The mission statement is published on NVAO's 
website. 

A description of the cultural and historical context of the work of NVAO is 
included in the separate context section. The Strategie Policy Statement clearly 
demonstrates the translation of the mission statements into a clearly formulated 
policy and management plan. The division of labour with relevant stakeholders in 
higher education is also described in the statement. The list of the main tasks of 
NVAO as well as other parts of the statement make clear that external quality 
assurance processes are major activities of the agency. The assessment 
frameworks confirm that NVAO employs a systematic approach to achieving its 
goals and objectives. 

Conclusion 

NVAO fully complies with ENQA criterion 4 / ESG 3.5. 
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4.5 ENQA criterion 5 / ESG 3.6: Independence 

Standard 

Agencies should be independent to the extent both that they have autonomous 
responsibility for their operations and that the conclusions and recommendations 
made in their reports cannot be influenced by third parties such as higher 
education institutions, ministries or other stakeholders. 

Key elements 

"Autonomy", "independence" 

The Panels finds this Standard is thoroughly described in the following guidelines. 
An agency will need to demonstrate its independence through measures, such 
as: 

• its operational independence from higher education institutions and 
governments is guaranteed in official documentation (e.g. instruments of 
governance or legislative acts); 

• the definition and operation of its procedures and methods, the nomination 
and appointment of external experts and the determination of the 
outcomes of its quality assurance processes are undertaken autonomously 
and independently from governments, higher education institutions, and 
organs of political influence; 

• while relevant stakeholders in higher education, particularly 
students/learners, are consulted in the course of quality assurance 
processes, the final outcomes of the quality assurance processes remain 
the responsibility of the agency. 

Evidence 

NVAO was granted the status of an autonomous administrative body with legal 
rights according to Dutch legislation. NVAO is accountable to the Committee of 
Ministers, which approves its budget, the annual report and the annual accounts. 
In accordance with the Treaty, the Committee of Ministers can only intervene in 
case of serious neglect by NVAO of its (initial) accreditation task, threatening the 
execution of that task. The Committee of Ministers can thus only intervene in the 
functioning of NVAO, but not in NVAO's decision-making. From the start of the 
accreditation system in the Netherlands and Flanders (2004, NL - 2005, FL) 
NVAO has been fully independent regarding decision making. 

Members of NVAO's Executive and General Board are mandated for four years by 
the Committee of Ministers. They are appointed in a strictly personal capacity 
and not as representatives of any organisation. 

In order to guarantee the independence of board and staff members, NVAO has 
ruled that members of both categories cannot participate in applications from 
institutions or programmes they have been associated with in any form during 
the previous five years. Members of the Board have to be completely 
independent in making decisions. If there is a specific application where this 
independence cannot be guaranteed, the member of the Board will withdraw 
from the decision-making process for the programme concerned. 

The legally binding assessment frameworks formulated by NVAO in consultation 
with relevant stakeholders define the standards which the assessments have to 
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refer to, the possible judgements in relation to each of the standards and the 
general conclusion that they may result in and how this conclusion should be 
reached (assessment rules), requirement regarding the composition of panels, 
the assessment processes and the decision-making by NVAO. The assessment 
frameworks for (initial) programme assessments also include a list of the 
documents that the programmes need to provide and guidance as regards the 
"threshold levels" for the different possible judgement that can be reached by an 
assessment panel. Finally the assessment frameworks include a chapter in which 
NVAO outlines the rules laid down by implementing regulations regarding 
conditional decisions and granting of improvement periods. 

Analysis 

The view of NVAO expressed in its SER is that the official status offers NVAO a 
good formal structure to guarantee its independence and it states that its 
independence is recognised by different stakeholders. The interviews with 
stakeholders during the site visit confirmed this statement. 

It is also evident to the Panel that NVAO has autonomous responsibility for its 
operations and its operational independence from higher education institutions 
and governments is guaranteed in official documents. 

The very detailed and legally binding assessment frameworks (as well as the 
continuous monitoring of the level of adherence to them) ensure that the 
appointment of external experts and the determination of the outcomes of the 
quality assurance processes are undertaken autonomously and independently 
from governments, higher education institutions, and organs of political 
influence. They also ensure that the conclusions and recommendations made in 
the reports cannot be influenced by third parties such as higher education 
institutions, ministries or other stakeholders and that the final outcomes of the 
quality assurance processes (the accreditation decisions) remain the sole 
responsibility of NVAO. 

Since the assessments of existing programmes are not carried out by NVAO, the 
Panel finds that the independence of the quality assurance agencies carrying out 
these types of assessments needs to be considered as part of the assessment of 
the operational independence of NVAO. In this regard, the Panel has noted that 
in the external assessments of both VHLORA and VLIR these agencies have been 
assessed as being fully compliant and the Dutch agency QANU as being 
substantially compliant with ESG 3.6 Independence and so do not raise questions 
as regards the independence of the processes carried out by these agencies. The 
situation is less clear regarding the other Dutch quality assurance agencies which 
were not (yet) vetted by ENQA, but NVAO explicitly requires their panel members 
to adhere to a Code of Conduct and to sign an independence and confidentiality 
declaration. 

Conclusion 

NVAO fully complies with ENQA criterion 5 / ESG 3.6. 
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4.6 ENQA Criterion 6 / ESG 3.7 External quality assurance criteria 
and processes used by the members 

Standard 

The processes, criteria and procedures used by agencies should be pre-defined 
and publicly available. 

These processes will normally be expected to include: 

• a self-assessment or equivalent procedure by the subject of the quality 
assurance process; 

• an external assessment by a group of experts, including, as appropriate, (a) 
student member(s), and site visits as decided by the agency; 

• publication of a report, including any decisions, recommendations or other 
formal outcomes; 

• a follow-up procedure to review actions taken by the subject of the quality 
assurance process in the light of any recommendations contained in the 
report. 

Key elements 

The Standard is self-contained, with the guideline adding on the need to ensure 
professional management and consistency and an appeals procedure. 

Evidence 

The standards (criteria), processes and procedures used by NVAO and the quality 
assurance agencies in the Netherlands and Flanders are defined in the publicly 
available assessment frameworks. 

All types of assessments include some form of self-assessment, an external 
assessment by a group of experts, including a student member, site visits and 
the publication of a report including the recommendations of the panel and (in a 
separate report) the decision by NVAO. 

A follow-up procedures is applied in some cases, in others not. The issue of 
follow-up is thoroughly described, analyses and concluded upon in section 4.1.6 
and will therefore not be repeated in this section and not influence the conclusion 
either. 

For the programme assessments carried out by the quality assurance agencies 
NVAO checks, as described in section 4.1.3, the quality of the assessment 
procedure based on the information provided in the assessment report and if this 
report leaves questions unanswered, NVAO requests additional investigation. 

To handle appeals NVAO has set up appeals commissions for the Netherlands and 
Flanders (competencies not given in the SER) that provides NVAO with advice on 
which NVAO can make a decision. NVAO is pleased with this set up although it 
reports further adjustments. If the decision on an internal appeal is negative, the 
institution can pass the appeal on to independent ministerial body in either the 
Netherlands or Flanders. 

Analysis 

The standards (criteria), processes and procedures used by NVAO and the quality 
assurance agencies in the Netherlands and Flanders are pre-defined and publicly 
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available and the processes all include the elements expected by Standard 3.7 
except a follow-up procedure which is not applied in all cases. 
Based on the numerous references to the assessment frameworks in the SER and 
not least in many of the interviews during the site visit, the Panel is convinced 
that NVAO and the quality assurance agencies pay careful attention to the 
declared principles of the different forms of assessment and that NVAO does its 
utmost to ensure both that the requirements and processes are managed 
professionally and that the conclusions and decisions are reached in a consistent 
manner. In relation to the issue of consistency the Panel wishes to point out 
what was mentioned in section 4.2.8 about clusters as clustering of assessments 
may also be beneficiary to consistency in assessments and decision making. 
Moreover, it is clear to the panel that NVAO is aware of the fact that as it is 
processing some 600 applications every year a constant attention to the 
consistency of the decision making is needed. 

The Panel relies upon the description of the appeals procedures. An appeals 
commission is mainly there to secure the rights of an institution. The judgements 
of the appeals commission may of course also teil something about the quality 
assurance and professionalism of NVAO. However; the Panel is in line with ESG 
and regards the appeals procedure as a part of ESG 3.7 (even if it is only given 
in the guidelines and not in the Standard itself). 

Conclusion 

NVAO fully complies with ENQA criterion 6 / ESG 3.7. 
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4.7 ENQA Criterion 7 / ESG 3.8: Accountability procedures 



Standard 

Agencies should have in place procedures for their own accountability. 

These procedures are expected to include the following: 

i. A published policy for the assurance of the quality of the agency itself, 
made available on its website; 

ii. Documentation which demonstrates that: 

• the agency's processes and results reflect its mission and goals of quality 
assurance; 

• the agency has in place, and enforces, a no-conflict-of-interest mechanism in 
the work of its external experts; 

• the agency has reliable mechanisms that ensure the quality of any activities 
and material produced by subcontractors, if some or all of the elements in its 
quality assurance procedure are subcontracted to other parties; 

• the agency has in place internal quality assurance procedures which include 
an internal feedback mechanism (i.e. means to collect feedback from its own 
staff and council/board); an internal reflection mechanism (i.e. means to 
react to internal and external recommendations for improvement); and 

• an external feedback mechanism (i.e. means to collect feedback from experts 
and reviewed institutions for future development) in order to inform and 
underpin its own development and improvement. 

iii. A mandatory cyclical external review of the agency's activities at least once 
every five years which includes a report on its conformity with the 
membership criteria of ENQA. 

Key elements 

This Standard is understood in its full text, there are no explicit key words as 
such in this very comprehensive formulation. However, the meaning of 
"subcontractor" is not made clear by ENQA. In various dictionaries a 
subcontractor is generally defined as an individual or in many cases a business 
that signs a contract to perform a part or all of the obligations of another's 
contract. In its decision making NVAO has to build upon quality assurance 
performed by other agencies as regulated by law. As far as the Panel sees it, 
ENQA's practise until now has not been to regards these as subcontractors, 
neither do we. 

Evidence 

i. A published policy for the assurance of the quality of the agency itself, 
made available on its website 

The "NVAO Quality Statement" of 2006 contains a section (section 4) presenting 
among other items NVAOs "policy" for its own internal quality assurance and the 
policy is also available on NVAO's website. 

ii. Documentation on specific issues 

As described in section 4.1.4, all of the Dutch assessments frameworks include a 
section describing how the independence (both factual and perceived) and also 
how the "no-conflict of interest" of the panel members is secured. The 
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mechanisms comprise different kinds of documents that each of the panel 
members and the secretary must sign. The Flemish frameworks do not contain 
anything similar. 

NVAO does not report that it uses subcontractors in the ordinary sense. Section 
4.1 describes how NVAO ensures the quality of the activities and material 
produced by the cooperating quality assurance agencies. To supplement these 
descriptions, it can be mentioned here that NVAO in its SER describes that it 
gives feedback to the quality assurance agencies and secretaries concerning the 
quality of the reports they produced on a regular basis. At the interviews during 
the site visit, the quality assurance agencies in both the Netherlands and 
Flanders confirmed that NVAO provides formal and informal feedback to them. 

The internal feedback and reflection mechanisms contain both written and verbal 
activities. In the SER, NVAO highlights the periodic meeting with the Advisory 
Council (three times a year), an extensive survey of the staff's satisfaction in 
2011 and yearly staff meeting to discuss the realisations of previous objectives 
and plans and formulate the objectives of NVAO's internal quality assurance. 

External feedback mechanisms comprises for instance regular meetings with the 
relevant ministerial representatives in the Netherlands and Flanders, the quality 
assurance agencies and different written evaluation of the procedures for initial 
accreditation 

iii. A mandatory cyclical external review of the agency's activities at least 
once every five years 

The documents governing the work of NVAO do not contain any requirements 
concerning external reviews of NVAOs activities, but with the 2007 review and 
the present 2012 review NVAO has so far been subject to an external evaluation 
every five years. 

Analysis 

A published policy for the assurance of the quality of NVAO exists and is available 
on the website of NVAO. 

The assessment frameworks for the Netherlands clearly reveal that NVAO has in 
place, and enforces, a no-conflict-of-interest mechanism in the work of the 
panels. The reviews of VLHORA and VLIR leaves the Panel convinced that a no- 
conflict-of-interest mechanism in the work of the panels is also in place and 
enforced in the assessments in Flanders. 

The quality assurance of the work of the quality assurance agencies is convincing 
and so are the internal and external feedback mechanisms. 

In general it is the impression from the SER and the site visit that NVAO has had 
a strong focus on developing its internal quality assurance since the external 
review in 2007 where critical points were raised. NVAO seems to have 
strengthened internal quality assurance, specifically the level of dialogue with 
stakeholders which was identified as a "weak" point in 2007, now seems to have 
developed and have become a "strong" point. During the site visit, a good 
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dialogue with stakeholders was repeatedly mentioned by external stakeholders 
as one of the characteristics of NVAO. 

Conclusion 

NVAO fully complies with ENQA criterion 7 / ESG 3.8. 
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4.8 ENQA criterion 8: Miscellaneous 

i. The Agency pays careful attention to its declared principles at all times, and 
ensures both that its requirements and processes are managed professionally 
and that its judgements are reached in a consistent manner, even if the 
judgements are formed by different groups. 

ii. If the Agency makes formal quality assurance decisions, or conclusions which 
have formal consequences, it should have an appeals procedure. The nature and 
form of the appeals procedure should be determined in the light of the 
constitution of the Agency. 

iii. The Agency is willing to contribute actively to the aims of ENQA. 
Key elements 

"consistent manner", "appeals procedure", "willing to contribute to the aims of 
ENQA" 

By "willing to contribute to the aims of ENQA" the Review Panel has looked upon 
the following two dimensions: 

1. Inwards: being a good role model by an active follow up of the 
recommendations given by the review panel in the previous review in 
2007 

2. Outwards: actively promoting ENQA/ESG on the European Arena for 
Higher Education 

Evidence and analysis 

Section 4.6 deals with the issues of "consistent manner" and "appeals 
procedures" and the analysis and conclusions related to these issues will not be 
repeated here but just form part of the frame of reference for the conclusion in 
relation to NVAOs level of compliance with ENQA membership criterion 8. 

The recommendations from the review in 2007 (September) was presented in 
that report, chapter 1.5 Recommendations. Two years later, in a letter to the 
ENQA-board, dated December 21, 2009, NVAO gives their response to the 
recommendations that they consider relevant for the organisation. NVAO SER of 
April 2012 contains a chapter 9 "Overview of actions taken since the previous 
external review 2007". Here NVAO deals both with recommendations regarding 
NVAO and recommendations regarding the system. Without going in details, it 
seems that NVAO has had a fairly good approach to this kind of follow up. The 
recommendation on developing the relationship between NVAO and the quality 
assessment agencies is dealt with on several occasions in the present report. 

It is the Panel's impression from the SER as well as its general knowledge about 
the international activities of NVAO, that NVAO has been actively engaged in the 
European arena and in that work actively contributed to the aims of ENQA. Just 
to mention: at present NVAO has an ENQA Board member, previously NVAO was 
the initiator for ECA, NVAO has been hosting several international conferences 
and a driving force behind international joint-projects. In short: NVAO is holding 
a high international profile. 

Conclusion 

NVAO fully complies with ENQA criterion 8. 
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5 Conclusion and recommendations 



5.1 Conclusion 

In the light of the documentary and oral evidence considered by it, the Panel is 
satisfied that, in the performance of its quality assessment functions, NVAO is in 
compliance with the ENQA Membership Regulations and in substantial compliance 
with the Standards and Guidelines for Quality Assurance in the European Higher 
Education Area. The Panel therefore recommends to the Board of ENQA that 
NVAO's Full Membership of ENQA be confirmed for a further period of five years. 

The Panel concludes that NVAO fully complies with the ENQA membership criteria 
2, 3, 4, 5, 6, 7 and 8 and substantially complies with the ENQA membership 
criterion 1. 



5.2 Recommendations 

In relation to its assessment of NVAO's compliance with the ESG, the Panel 
recommends that NVAO: 

• makes the link between the NVAO standards and ESG Part 1 more explicit in 
the assessment frameworks; 

• refines the descriptions of the aims and objectives, ensures that they are 
prominent in the frameworks and shows how the various elements of the 
frameworks contribute to the aims and objectives; 

• formulates explicit and public criteria about how it reaches its accreditation 
decisions in relation to the programme assessments in all cases; 

• establishes a clear procedure on how to handle cases where the conclusions 
in the assessment report are not be accepted by NVAO; 

• strengthens the predictability of the timeframe and efficiency of its decision- 
making process; 

• further strengthens the focus on quality improvement and enhancement of 
the HEI's; 

• clarifies a) the purpose of every kind of report, b) the readership and c) the 
needs of the various kinds of readers in order to enhance the readability; 

• gives a high priority to a)the identification of the interested parties for 
system-wide analyses and of their needs b) the definition of a realistic 
schedule of system-wide analyses and c)the production of system-wide 
analyses corresponding to the needs which were identified. 
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Extemal review of the Accreditation Organisation of the Netherlands and Flanders 
(NVAO) by the European Association for Qüality Assurance in Higher Education (ENQA) 

Annex I: TERMS OF REFERENCE 
February 2012 



1 Background and Context 

NVAO in de pen den tl y ens ures the quality of higher education: by assessing the quality 
control of higher education institutions in the Netherlands and accrediting higher 
educations programmes in Flanders and the Netherlands. Inaddition, NVAO entourages 
quality awareness within higher education institutions and advances higher education in its 
national and international context. 

The organisation was established in 2005 by treaty between the Netherlands and Fla nders 
as an independent quality assurance agency. 

NVAO was reconfirmed Full membership of ENQA on 20 December 2007 as a result of a 
review coordinated at national level. 

2. Purpose and Scope of the Evaluation 

This is a type A review, as defined in the Guideünes for externa! reviews of quality 
assurance agsnciss in the European Higher Education Area. It will evaluate the way in 
which and to what extent NVAO fulfils the criteria for the ENQA membership and thus the 
Standards and Guideünes for Quality Assurance in the European Higher Education Area 
(ESG). Consequently, the review will also provide information to the ENQA Board to aid its 
consideration of whether NVAO should be reconfirmed Full Member of ENQA. The review 
panel is not expected. however. to make any judgements as regards granting NVAO Full 
Membership in ENQA. 

In addition to analysing NVAO compliance with the ENQA membership criteria, and thus 
with the ESG, the review aims to give a first external re~lection on the new accreditation 
system in the Netherlands. 

3. The Review Process 

The process is designed in the light of the Guideünes for external reviews of quatity 
assurance agsnciss in the European Higher Education Area. 

The evaluation procedure consists of the following steps: 

» Formulation of the Terms of Reference and protocol for the review, 

• Nomination and appointment of the review panel; 

* Self-evaluation by NVAO including the preparation of a self-evaluation report; 
» A site visit by the review panel to NVAO; 
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• Preparation and completion of the final evaluation report by the review panel, 

• Scrutiny of the final evaluation report by the Review Committee of the ENQA Boarc; 

• Analysis of the scrutiny by the ENQA Board and their decision regarcing ENQA 
membership; 

• Foilow-up of the panel's and/or ENQA Board's recom mendations by the agency. 

3.1 Nomination and appointment of the review team members 

The review panel consists of five members: Four external reviewers (one or two quality 
assurance experts, representative(s) of higher education institutions, student member) 
and a review secretary. Three of the reviewers (including the review secretary) are 
nominated by the ENQA Board on the basis of proposals submitted to ENQA by the 
national agencies, and are drawn from senior serving members of Board/Council or staff of 
ENQA member agencies. The fourth external reviewer is drawn from a nomination 
provided by the European University Association (EUA) The nomination of the student 
member is asked from the European Students' Union (ESU). One of the panel members 
serves as the chair of the review. 

Current members of the ENQA Board are not eligible to serve as reviewers. 

ENQA will provide NVAO with the list of suggested experts with their respective curricula 
vitae to establish that there are no known con'licts of interest. The experts will have to sign 
a non- conflict of interest statement as regards the NVAO review. 

3.2 Self-evaluation by NVAO, including the preparation of a self-evaluation report 

NVAO is responsible for the execution and organisation of its own self-evaluation process 
and shall take into account the following guidance: 

• Self-evaluation is organised as a project witha clearly defined Schedule and includesall 
relevant internal and external stakeholders; 

• The self-evaluation report is broken down by the topics of the evaluation: backgrounc 
description of the current situation of the Agency; analysis and appraisal of the current 
situation, proposals for improvement and measures already planned; a summary of 
perceived strengths and weaknesses; 

• The report is well-structured, concise and comprehensively prepared. It clearly 
demonstrates the extent to which NVAO fulfils its tasks of external quality assurance 
and meets the criteria for the ENQA membership and thus the ESG. The report is 
submitted to the review panel a minimum of four weeks prior to the site visit. 

3.3 A Site Visit by the Review Panel 

NVAO will draw up a draft proposal of Schedule of the site visit to be submitted to the 
review panel at least three months before the planned dates of the visit. The Schedule 
includes an indicative timetable of the meetings and other exercises to be undertaken by 
the review panel during the site visit, the duration of which is 2 days. The approved 
Schedule shall be given to NVAO at least two months before the dates of the visit, in order 
to properly organise the requested interviews. 
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The review panel will be assisted by NVAO in arriving to The Hague. Netherlands 

Where appropriate, the site visit will close with an oral presentation and discussion of the 
major issues of the evaluation between the review panel and NVAO. 

3.4 Preparation and completion of the final evaluation report 

O n the basis, of the review panel's f indings, the review secretary will draft the report in 
consultation with the review panel. The report will take into account the purpose and scope 
of the evaluation as defined under article 2. It will also provide a clear rationale for its 
findings with regards to each ENQA membership criteria. A draft will be submitted for 
comment to NVAO within two months of the site visit for comment on factual accuracy. If 
NVAO chooses to provide a statement in reference to the draft report it will be submitted 
to the chair of the review panel with in two weeks after the receipt of the draft report. 
Thereafter the review panel will take into account the statement by NVAO. f inalise the 
document and submit it to NVAO and ENQA. 

The report is to be finalised within three months of the site visit and will not exceed 40 
pages in length. 

4. Follow-up Process and Publication of the Report 

NVAO will consider the expert panel's report and inform ENQA of its plans to implement 
any recommendations contained in the report. Subsequent to the discussion of the 
evaluation results and any planned implementation measures with ENQA, the review 
report and the follow-up plans agreed upon will be published on the NVAO website. 

5. Budget 

NVAO shall pay the following review related fees: 

- Chair: 5 r OOO EUR 

- Review secretary: 5,000 EUR 

- Other panel members (3,000 EUR for each of the three panel members): 
9 p OOO EUR 

- Administrative overhead for ENQA Secretariat: 5.000 EUR 

- Travel and subsistence expenses (approximate): 6..000 EUR 

This gives a total indicative cost of 30.000 EU R for a review team of 5 mem bers. In the 
case that the allowance for travel and subsistence expenses is exceeded, NVAO will cover 
any additional costs after the completion of the review. However, the ENQA Sec reta riat will 
endeavour to keep the travel and subsistence expenses in the limits of the planned budget, 
and will refund the difference to NVAO if the travel and subsistence expenses go under 
budget. 
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6. Indicative Schedule of the Review 

The duration of the evaluation is scheduled to takeabout IO months, from January 2012 to 
October 2012: 

NVAO begins self-evaluation October 2011 

Agreement on terms of reference and protocol for review January 2012 

Appointment of review panel members February 2012 



Preparation of site visit Schedule and indicative timetable March 2012 

NVAO self-evaluation completed end April 2012 

Briefing of review panel members end May 2012 

Review panel site vis it end June 2012 

Draft of evaluation report to NVAO August 2012 

Statement of NVAO to review panel if necessary August 2012 

Submission of final report to NVAO and ENQA September 2012 
Consideration of the report by ENQA and response 

of NVAO September 2012 

Publication of report and implementation plan October 2012 
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Director of Department of Quality Assurance, NOKUT (Norwegian Agency for 
Quality Assurance in Education) 
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2004 Institutional Accreditation and accreditation of Master in 

Hydrogeology, Chairing the Expert Panel commissioned by 
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2002 International Comparative Evaluation of BSc programmes in 

Agricultural Science. Commissioned by The Danish Evaluation 
Institute. Chair of Panel. 

Signe Ploug Hansen 

Secretary of the Panel 

Director of Methodology and coördinator of IQA (internal quality assurance) at 
EVA (the Danish Evaluation Institute) 

Member of the Advisory Board of the evaluation unit at Malmö Högskola, 
Sweden. 

2012 External review of the National Agency for Higher Education in 

Sweden. Panel member 
2007-2011 Initiator of the ENQA Internal quality assurance group (the 

IQA group) and leading member of its steering group 
2002 International Comparative Evaluation of BSc programmes in 

Agricultural Science. Secretary of the panel. 



Julian Tobias Hiller 

Student member of the Panel 

Student of Mechanical Engineering at the Leibniz University of Hannover, 
Germany. Integrated Studies, now at a Masters level. 

Member of the Accreditation Council of Germany 
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improve the quality of higher education). 

1972-2006: professor of computing science and engineering at the Université 
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informaties/computer science university programs in Flanders 
2011-2012: member of an external review panel for AEQES: assessment of the 

informaties/computer science programs in the French-speaking part 

of Belgium 

2011-2012: member of an AEQES working group in charge of proposing new 
criteria for quality assessment in the French-speaking part of 
Belgium. 
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company of Université Catholique de Louvain 

Norma Ryan 

Member of the Panel 

Director of the Quality Promotion Unit, National University of Ireland Cork - 
University College Cork, Ireland 

• Member of the Irish Universities Association Quality Committee 

• Member of the Irish Higher Education Quality Network 

• Member of the Irish National Bologna Experts Team 

2012: Representative of the EUA and Chair of the Steering Committee for the 
European Quality Assurance Forum 

2008: External review of Letterkenny Institute of Technology, Ireland. Panel 
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2010: External institutional review of Tallaght Institute of Technology, Ireland. 
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2011: external institutional review of American College Dublin, Ireland. Panel 
Member. 

2011: Review of Academie Programmes, KTH, Stockholm, Sweden: external 
reviewer 

2011: Institutional review of St Augustine Catholic University, Tanzania. Panel 
Secretary 
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C: Documents considered during the review 



(Listed in order of importance for the assessment) 

• Self-evaluation report NVAO 2012 

• Self-evaluation report NVAO 2012 - Attachments 

• Report of the committee for the review of the Accreditation Organization of 
The Netherlands and Flanders (NVAO) (September 2007) 

• The Dutch website of NVAO: http : //www. nvao. net/ 

• The English website of NVAO: http : //www .nvao . net/ 

• Letter from the Chairman of NVAO to the president of ENQA about The follow- 
up of NVAO on recommendations included in the external review review of 
NVAO 2007 (December 21 2009) 

• The NVAO Quality Statement of 2006 (handed out at site visit) 

• Diagram illustrating the different forms of assessments within the new 
accreditation system in the Netherlands and the relations between these 
forms (handed out at site visit) 

• Tables containing comparisons between the accreditation framework in 
Flanders and the Netherlands and the ESG (handed out at site visit) 

• Report of the External Review of the Flemish Council of University Colleges 
(VLHORA) (October 2008) 

• Report of the committee of the review of The VLIR Quality Assurance Unit 
(May 2009) 

• External review of the agency Quality Assurance of Universities in the 
Netherlands (QANU) (October 2010) 
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NVAO, the Accreditation Organisation 
forthe Netherlands and Flanders 
[Nedertands-Vtaamse Accreditatie- 
organisatie] was formally established by 
the Dutch and Flemish governments as a 
binational organisation on 1 February 
2005. The Netherlands and Flanders decided 
to found a binational organisation to 
express the cross-border cooperation on 
external quality assurance in the spirit of 
the Bologna Declaration. In this regard it is 
a significant advantage that the Netherlands 
and Flanders share the Dutch language. 

The main objectives of the review scheduled for 201 2 are: 

1 . Renewal of NVAO's full membership of ENQA, by establishing 
the agency's compliance with the European Standard and 
Guidelines 2009 (ESG), parts II and III, includingtheterms 
of reference of ENQA (review A), as well as the agency's 
compliance with the equivalent ESG requirements of EQAR, 
leading to the extension of NVAO's EQAR-registration. 

2. A first reflection on the new accreditation system in the 
Netherlands that came into force on 1 January 201 1, 
which constitutes an additional request of NVAO to ENQA 
and the review committee; 

3. To contribute to NVAO's internal quality monitoring with 
regards to its organisation and its operation. 

Accordingly, NVAO evaluates itself in this self-evaluation 
report from three angles: 1. with reference to the ESG, 2. from 
the point of view of introduction of the new accreditation 
system in the Netherlands and, 3. with regards to its func- 
tioning organisation. This three-sided evaluation is distributed 
among the chapters of the report in the following manner: 

> Chapter 1 (partly), 3, 4 and 5 (partly) deal mainly with 
compliance with ESG, respectively Parts 2 and 3; 

> Chapter 7 deals primarily with the new accreditation system 
in the Netherlands; 

> Chapter 1 and 5 describe and evaluate NVAO and its role in 
the new accreditation system. 

The remaining chapters 2, 8 and 9 describe the systems of 
higher education in Netherlands and Flanders and summarise 
NVAO's activities, referringto the more evaluating chapters 
mentioned above. 

The following introductory remarks are meant to put the three 
main objectives of this self-evaluation report in perspective: 
1. NVAO is subject to a five-yearly external review in order to 
renew its full membership of the European Association for 



Quality Assurance in Higher Education (ENQA). In 2007, 
NVAO successfully passed a first review of ENQA and 
became full ENQA-member.The present review of 201 2 is 
the second of its kind. The previous review was initiated 
on a binational request by the ministers of Education in 
Flanders and the Netherlands. Since November 2008, 
NVAO has been registered in the European Quality 
Assurance Register for Higher Education (EQAR), the 
register for quality assurance agencies (QAA's) that review 
institutions or programmes in higher education on a 
regular basis and substantially comply with the ESG. 

2. The previous review in 2007 had as secondary objectives 
the evaluation of the - then- new accreditation systems in 
the Netherlands and Flanders and an assessment of their 
differences. Because the new system in the Netherlands 
has only been active from 201 onwards, the present review 
only entails a first reflection of the new regime. Notwith- 
standing the different objectives and conditions of the two 
reviews, chapter 9 shortly reports the measures taken in 
response to the recommendations in the 2007 review. 

3. Internal quality assurance is an important issue in NVAO's 
day to day operation, which consists of processing some 
600 applications every year. This task calls for constant 
attention to the consistency of the decision making and 
continuous improvement of processes. 

NVAO sees the external review as an extra incentive for 
its own quality assurance and forthe improvement of its 
internal and external operational processes. 

NVAO is a binational institution.The differences between the 
Dutch and Flemish systems in higher education and external 
quality assurance are relatively small, but nevertheless 
considerable in a few aspects.This self-evaluation report was 
written from the perspective of the many similarities between 
the systems of higher education and accreditation in both 
countries. Only when necessary for a proper understanding, 
it refers to the differences between these systems. 
One important difference regards the timetable forthe 
renewal of the accreditation regimes: while the new system 
has been in force in the Netherlands since 1 January 201 1 
and is being further optimized at the present moment, the 
new accreditation system in Flanders is in its final design 
stage and will be in force from the academie year 201 3-2014. 

In setting up its internal quality insurance system, NVAO iden- 
tified its most important internal and external stakeholders 
and organised regular consultations with them, individually, or 
through umbrella organisations. An inventory of the results of 
these consultations was drafted duringthe preparation of this 
self-evaluation report, and is included in the document. The 
NVAO also consulted various stakeholder groups on different 
occasions on the present report. Eventually, the report was 
approved bythe General Board of the NVAOon 16 April 2012. 



Introduction and aims of the external review / 7 



Chapter 
1 

Summaj 



and ga 



y 



«hera. 



apog 



aisa. 



On 1 February 201 2 NVAO off icialLy existed seven years. 
Since its foundation NVAO has carried out the primary 
assignment as defined in theTreaty between the Dutch and 
Flemish Education ministers of 2003, which is described as: 

The primary assignment of NVAO is to accredit existing 
programmes in higher education (accreditation) and to assess 
new programmes (initial accreditation) in the Netherlands and 
Flanders. NVAO can be requested to carry out additional tasks 
by the Ministers of higher education in both countries if 
these assignments support or suppiement NVAO's primary 
assignment. Secondly NVAO is assigned to play an important 
mie in internationalisation. 

In extension of this primary assignment NVAO formulated its 
first mission and strategy in 2006 and renewed it in 201 2. 
Since 2005 NVAO has completed a large number of applica- 
tions for (initial) accreditation in the Netherlands (NL) and 
Flanders (FL), parallel to a substantial international task 
in Europe and world-wide. Several additional tasks in the 
Netherlands and Flanders have also been taken up. Up to 
1 January 201 2 NVAO decided (or in some cases advised) 
on more than 4.000 applications of all kind. 

The questions this review sets out to answer are: 

> whether NVAO, with this substantial effort, successfully 
carried out its primary assignment and; 

> whether NVAO operates effectively with regard to its main 
stakeholders, being both governments.the institutions in 
higher education and the students in higher education in 
the Netherlands and Flanders. 

For this purpose NVAO's Strategie Policy Statement 
emphasizes the importance of: 

> both robust quality assurance and quality enhancement 
based on expert and objective judgements; 

> the further development of a quality culture in higher 
education and; 

> reinforcement of the Dutch-Flemish cooperation, as well 
as cooperation in the European higher education area. 

What is the current position of NVAO regarding these 
objectives and challenges? 

In the past years NVAO has built a strong and respected 
position in higher education in the Netherlands and Flanders 
and Europe. 

> NVAO plays a leading, independent and critical role in quality 
assurance in higher education in the Netherlands and 
Flanders and was the initiator of the development of the 
new accreditation system in the Netherlands and Flanders; 



> NVAO proved to be a very f lexible organisation capable of 
completinga large number of additional tasks on request of 
the Ministers of Education in the Netherlands and Flanders; 

> NVAO communicates and cooperates with its main stake- 
holders in an open and transparent manner, with the aim of 
contributing in the best possible way to the further develop- 
ment of a quality culture in higher education and thus to the 
quality of higher education in the Netherlands and Flanders; 

> Internationally, NVAO contributes considerably in Europe 
through ENQA, ECA, EQAF and European projects, as well as 
worldwide, through its involvement in INQAAHE. 

Besides these achieved results there are several issues 
NVAO should pay attention to in the next years to improve its 
operation and the operation of the accreditation system in 
the Netherlands and Flanders. 

> The administrative burden of the accreditation system 
should be reduced by focusing on the essence of quality 
on institutional and study programme level; 

> The large number of applications necessitates permanent 
attention to consistency in decision making; 

> Additional tasks should be planned better and formal 
deadlines should be kept more rigorously; 

> NVAO should remain self-critical and self-reflective with 
regards to (systems of) quality assurance and its own 
functioning, by staying first and foremost focused on 
quality and its enhancement; 

> The binational nature of NVAO should contribute stronger 
to Dutch-Flemish cooperation in higher education; 

> International projects and activities should be more 
benef icial for institutions in higher education and/or 
NVAO as a whole. 

According to its own assessment, NVAO fully complies with 
the ESG of ENQA, even if there are several issues that require 
its permanent attention. A major challenge for NVAO is the 
successful implementation of the new accreditation systems 
in the Netherlands and Flanders and to make these systems 
converge as much as possible. A first evaluation of the new 
system by the review committee is a very useful step in this 
process. 

Another permanent issue is the consistency of NVAO's internal 
processes. In the past years NVAO focused in its internal 
quality assurance on a more effective steering of the quality of 
its internal processes. The present external review by ENQA 
will also be used to improve NVAO's processes. 
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This self-evaluation report refers to the 
years 2007-201 1 , a period which is 
characterised by considerable changes 
in the accreditation systems used in 
Flanders and the Netherlands. A brief 
overview of these systems is given here to 
offer the reader a framework for the next 
two chapters, which describe how NVAO 
meets the European Standards and 
Guidelines within the accreditation 
systems of the Netherlands and Flanders. 

2.1 / The 'old system' 

In the Netherlands, the first accreditation system was 
operational from 2003 to 201 0, in Flanders from 2005 to 201 2. 
So the Netherlands has already made the transition to the 
new system, and Flanders will follow in 201 3. To avoid any 
ambiguity, the current Flemish system will already be referred 
to in the past tense, although it is still operational in 201 2. 

The previous system was layered: NVAO had to base its 
accreditation decisions on assessment reports compiled 
by quality assurance agencies. In the Netherlands these 
were free-market agencies, known as VBIs, which had to be 
assessed annually by NVAO and when appraised positively, 
were entered on a public list. Higher education institutions 
were then free to use their services. In Flanders, on the other 
hand, the umbrellas of universities (VLIR) and university 
colleges (VLHORA) were designated by law to act as quality 
assurance agencies. In Flanders programmes had to be 
assessed in clusters, in the Netherlands there was no such 
requirement and universities of applied sciences and private 
institutions therefore chose not to do so; the universities, 
however, did. 

The VBIs and quality assurance agencies were authorised 
to compose panels (following rough guidelines set by NVAO) 
and followed the accreditation framework as defined by law. 
The framework comprised six themes: 

> the programme's objectives 

> curriculum 

> staff 

> facilities 

> internal quality assurance 

> results. 



These six themes were subdivided into 21 standards that were 
assessed on a four-point scale (satisfactory, unsatisfactory, 
good and excellent). Forassessingattheme level a dichoto- 
mous scale was used (satisfactory or unsatisfactory), as was 
the case for the f inal assessment. So NVAO could not grant 
any conditional accreditations. In Flanders, in the event the 
final conclusion reached was 'unsatisfactory', a recovery 
period could be granted by the Flemish Government whereas 
in the Netherlands the recovery option was a theoretic 
possibility, which in practice, however, turned out not to be 
a realistic one. 

For new programmes, the panels were composed by NVAO 
(with some exceptions in the Netherlands). Virtually the 
same assessment framework was applied, with the exception, 
of course, for the theme results, which was replaced with that 
of 'conditions for (financial) continuity'. 

2.2 / The evaluation of the 'old system' 

In Flanders and the Netherlands a great deal of attention has 
been paid to the functioning of the accreditation system. Based 
in part on evaluations and discussions with stakeholders, NVAO 
carried out a self-evaluation in 2008 to prepare the ground for 
the design of a new system, for which the preparations started 
that year. Positive elements of the previous system were: 

> the accreditation system was introduced smoothly and was 
fit for purpose: all programmes were assessed as planned 
during the first phase of the system; 

> the system has laid a quality foundation forthe sectors of 
higher education in Flanders and the Netherlands. To put it 
bluntly: all the wrecks have been taken off the road; 

> the attention for quality assurance has strongly increased; 

> internationally, the system is understood and seen as a 
solid one. 

These positive points are offset by the following negative ones 
that emerged from the evaluation: 

> the fear for negative assessments (and subsequent loss of 
prestige or closure of a programme) has led to programmes 
and panels giving and asking (too) much information; 

> the focus has been too much on procedures and processes, 
and too little on the content of the programmes and the 
achieved learning outcomes; 

> teaching staff had difficulty seeing themselves as part of the 
system; 

> the large amount of available public information (assess- 
ment reports, accreditation decisions) has paradoxically led 
to 'an inaccessibility of information'; 

> In the Netherlands, in the absence of an obligation to assess 
in clusters, the comparative quality of the assessments has 
decreased. 
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This analysis was broadly shared by and with stakeholders 
and resulted in Flemish and Dutch ministers, in charge of 
higher education, commissioning the design of a new 
system that was to largely meet the points of criticism as 
described above.The ministers put great emphasis on a 
necessary reduction of the bureaucratie burden, without 
any concessions being made to the international legitimacy 
of the system. 

2.3 / The 'new system' 

In the new system, which was introduced in the Netherlands 
on 1 January 201 1 and in the design that is currently being 
discussed in Flanders and which is expected to make it 
through the Flemish parliament before the summer, 
programme accreditation remains at the heart of the system. 
In both countries, however, an institutional audit is added 
to the system with a view to realising an important adminis- 
trative saving at programme level.The institutional audit 
concerns the assessment of the institution's quality 
assurance system and ultimately leads to a judgement on 
the question whether an institution is in control of the quality 
of its education. In the Netherlands, the outcome of the audits 
will have immediate consequences, whereas Flanders regards 
the audits, which will be carried out in 201 5 and 201 6, as a 
baseline measurement that is obligatoryforall institutions. 
In the Netherlands the audit comprises five standards: 

> vision of the institution regardingthe desired quality of its 
education; 

> the policy developed and the means used to realise this 
vision; 

> a management information system; 

> improvement capacity and measures; 

> structure and culture of the organisation and its decision- 
making process. 

In Flanders, the discussion about the content of the institu- 
tional audits is still ongoing, but both systems will broadly use 
the same standards. The wide Flemish scope of the institu- 
tional review, ranging f rom the institution's policy to internal 
quality assurance by focusing on the quality of education, 
can also be found in the first Dutch institutional reports. 
In both systems the audits can have three possible outcomes: 
satisfactory, conditionally satisfactory and unsatisfactory. 
In the Netherlands, a final conclusion that is either 
'satisfactory' or 'conditionally satisfactory' means that 
the assessment of the programmes follows the framework 
of the so-called 'limited programme assessment'. 
This framework, which in Flanders will be applied to all 
programme assessments, has only three standards: 



> the programme's objectives and qualifications; 

> learning content and learning environment; 

> assessment procedures, examination and achieved 
learning outcomes. 

If an institution in the Netherlands fails the institutional audit 
or if it does not want to participate in it for motivating reasons 
(for example, the limited size of an institution), the 
programmes will be assessed on the basis of the framework 
of the 'extensive programme assessment'. This framework 
strongly resembles the 'old framework' as described above. 
h Flanders, the institutional audit will be made obligatory 
for all official institutions anyway, so that the programme 
assessment will always be a limited one. For non-official 
institutions, the Standard of institutional internal quality 
assurance will be added to the programme assessment. 

A number of changes have been made to the new system 
in response to a number of negative experiences with the 
previous one: 

> NVAO is involved in the composition of the panels before- 
hand by means of explicit approval (the Netherlands) or 
advice (Flanders); 

> the Netherlands, too, now has an effective recovery period; 

> the standards of the programme assessments are judged on 
a four-point scale; in the Netherlands, this is also the case 
for the panel's final conclusion regarding the overall quality 
of the programme. In Flanders, the discussion on this matter 
is still ongoing; 

> the number of documents that programmes are required 
to make available has been reduced considerably; 

> the reports have to contain a public-friendly summary, 
which NVAO can use to provide information to the public; 

> the secretaries of the programme panels will receive 
training; 

> in the Netherlands, assessments in clusters will become 
obligatory as of 2013. 

The experiences with the new system are, of course, still 
limited. Chapter 7 of this SER contains a first and provisional 
evaluation of the functioning of the new Dutch system. 
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The review panel is requested to assess 
on the basis of the following chapters 
whether and to what extent NVAO 
complies with the ESG.The assessment 
concerns NVAO's compliance regarding 
the individual standards and guidelines of 
ESG, as well as an overall judgement. 



3.1 /Use of internal quality assurance 
procedures 

External quality assurance procedures should take into account 
the effectiveness of the internal quality assurance processes 
described in Part 1 of the European Standards and Guidelines. 

Facts and findings 

The accreditation system in the Netherlands is a combination 
of an institutional audit (IA) and programme accreditation 
(PA), based on Limited (LPA) orextended programme assess- 
ment (EPA).The current system in Ftanders consists of 
programme accreditation, based on extended programme 
assessment, but a new accreditation system, containing both 
IA and LPA, is under construction and will be implemented in 
the academie year 201 3/201 4. An extended description of 
(the evolution of) the Dutch and Flemish accreditation system 
is included in chapter 7. 

Table 1 



Part 1 of the ESG refers to the following procedures in the 
institutions for higher education: 

(1) Policyand procedures for quality assurance 

(2) Approval, monitoring and periodic review of programmes 
and awards 

(3) Assessment of students 

(4) Quality assurance of teaching staff 

(5) Learning resources and student support 

(6) Information systems 

(7) Public information. 

AU these quality aspects are included in the NVAO frameworks 
at institutional and/or programme level. Table 1 shows which 
standards of the frameworks cover these respective aspects. 
The frameworks LPA and IA complement each other, which is 
described in chapter 2 and will be explained further in 
chapter 7. As the standards of the former Dutch accreditation 
framework are almost similarto those of the present EPA-NL, 
and as the standards in the present Flemish framework are 
still those at programme Level, columns IV and V also coverthe 
standards used in the period 2004-201 - the first phase 5> of 
the accreditation system in the Netherlands. 
A major building block of the new accreditation system is the 
way LPA and IA complement each other. Process oriented 
aspects that programmes have in common or are organised 
at a higher level, such as facilities and systems of internal 
quality assurance, are (NL) or will be (FL) assessed once in 
the institutional audits and not anymore repeatedly at 
programme Level. This should lead to a 25% reduction of the 
administrative burden involved in programme assessments. 



II Framework IA -NL 1 III Framework LPA - NL 2 IV Framework EPA - NL 3 " V Framework - PA - FL 4 



> Policy and procedures for quality 
assurance 

> Approval, monitoring and periodic review 
of programmes and awards 

> Assessment of students 

> Quality assurance of teaching staff 

> Learning resources and student support 

> Information systems 

> Public information 



Standard 4 



13,14and 15 

13, 14and 15 
16 

8 

11 and 12 
12 
12 



2, Assessment 
and testing 
3, Quality of Staff 
4 
4 
4 



1 > Framework Institutional Audit (IA)- the Netherlands, attachment 11. 
2> Framework Limited Assessment of Programmes (LPA) - the Netherlands, 
attachment 11. 

3> Framework Extended Asssessment of Programmes (EPA) - the Netherlands, 
attachment 11. 

4> Framework Programme Accreditation (PA) - Flanders, attachment 11. 



5> The initial phase spans six years in the Netherlands and eight years in 
Flanders, during which all programmes will go through an assessment 
under the new system. 



Analysis of Standards and Guidelines part 2/15 



Self-evaluation 

The NVAO frameworks cover all mentioned aspects of internal 
quality assurance of institutions in higher education.The new 
frameworks encourage the development of quality assurance 
and of a quality culture within the entire institution.The IA frame- 
work focuses on the internal quality assurance (IQA) processes 
at institutional level and the processes related to individual 
programmes offered bythe institution.The LPAframework 
focuses on the quality of the content and the achieved 
learning outcomes of the programme. Whereas the previous 
frameworks employed a more technical approach, the new 
system allows for a more thorough discussion about system- 
atic quality assurance and quality culture at institutional level 
and the quality of content and learning in individual programmes. 
Most higher education institutions in the Netherlands 
applied for IA in 201 1 and thus will be able to benefit from 
the reduction in administrative burden in LPAs in the coming 
years. Results from the first lA's indicate that the new frame- 
work encourages the development of quality assurance at an 
institutional level. However, smaller institutions are not likely 
to benefit from the IA and will therefore continue to operate 
underthe regime of the EPA. 

In Flanders, LPA and IA will be compulsory elements in the 
new accreditation system for all institutions. EPA will not be 
applied any longer. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> The procedure for the lAconsists of an overall assessment 
of the procedures within an institution, combined with 
several audit trails. These audit trails monitor the way 
institutional measures and regulations are perceived and 
acted upon at programme level (top down) and how the 
institutional IQA system operates in case of concrete quality 
problems (bottom up).The first accomplished audits show 
encouraging results in this respect: institutions realisethat 
they can only benefit from the new system when their 
internal systems are coherent and interactive at both levels. 
NVAO should monitor whether this interaction is demon- 
strated convincingly in the IA. 

3.2 / Development of external quality 
assurance processes 

The aims and objectives of quatity assurance processes 
should be determined before the processes themselves are 
developed, by all those responsible (including higher education 
institutions) and should be published with a description of 
the procedures to be used 



Facts and findings 

Both the previous and the current frameworks were designed 
in consultation with representatives of institutions and 
programmes, quality assurance agencies and other assess- 
ment experts. The aims and objectives of the systems were 
discussed with stakeholder groups and the frameworks were 
tested before the system came into force.The description of 
the drafting of the current frameworks in 2008 and 2009 in 
chapter 7 outlines this process.The frameworks were 
discussed in parliament and are published bythe government 
and NVAO. Additional information on the procedures is 
provided in guidelines.The frameworks were made available 
online on NVAO's website well before the start of the systems 
in both countries (www.nvao.net). 

The stakeholders and NVAO refrained from drawing up overly 
detailed or excessively normative or prescriptive frameworks. 
The new frameworks were conceived to accommodate the 
diversity of higher education programmes in the Netherlands 
and Flanders, and to respect the autonomy of institutions. 
The pilots of 2009 showed that stakeholders had a favourable 
attitude towards the new system. The development, imple- 
mentation and operation of the new accreditation system is 
being monitored by resonance groups representingthe impor- 
tant stakeholders in both Flanders and the Netherlands. 

Self-evaluation 

Initially, the institutions of higher education expressed their 
preference for a system of institutional accreditation only to 
replace the previous programme accreditation. A proposal 
to this extent was rejected by NVAO and the Dutch parliament. 
Eventually, the institutions accepted the new frameworks as 
a challenging innovation and actively took part in the develop- 
ment of the new system by participating in the pilots. This 
stakeholder involvement can be seen as a good practice. 
NVAO is convinced that the new system provides an effective 
compromise between firm quality assurance on programme 
level through programme accreditation, on the one hand, 
and reduction of the administrative burden through the 
institutional audit, on the other hand. The system allows for 
the enhancement, profiling 6 ', and differentiation of individual 
programmes by applying for distinctive (quality) features. 
The profiling is further enhanced bythe introduction of the 
use of a four-point scale for quality judgements on the stand- 
ards of the framework (NL-FL) and in the overall conclusion. 



6> Profiling is related to the mission, strategy and characteristics of an insti- 
tution or programme. Differentiation is related to the level and quality of a 
programme. 
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Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> It is vital that NVAO remains in close contact with the 
stakeholders, especially the representing bodies of the 
higher education institutions, to evaluate the experiences 
and to monitor whether the system functions as intended, 
especially in the context of announced, new regulations by 
the Dutch government. Also in Flanders this is essential 
since the decree specifies two evaluations of the new 
accreditation system while being implemented.This 
dialogue should be open and done in full independence. 

3.3/ Criteria fordecisions 

Any formal decisions made as a result ofan external quality 
assurance activity should be based on explicit published 
criteria that are applied consistently. 

The frameworks for EPA, IPA and IA use clearly distinguishable 
standards and a four-point scale (excellent, good, sufficiënt, 
insufficiënt), both for the assessment of the programme as a 
whole and for the separate standards. The underlying criteria 
and decision rules for each of these judgements are clearly 
outlined in the frameworks and explained in guidelines 
attached to the frameworks. The introduction of the recovery 
period in the Netherlands gives panels more leeway to pass a 
negative judgement when necessary, without causing a forced 
shutdown of a programme. 

In the current Flemish frameworks the four-points scale is 
only applied to the sub-standards; here also the four- point 
scale is explained in guidelines. The recovery period has been 
part of the Flemish accreditation system since the start in 2005. 

The NVAO has put in place the following measures to safe- 
guard the consistent application of the criteria during the 
assessment of applications for accreditation: 

> Approval of panels bythe NVAO: in the Netherlands NVAO 
approves panels according to the criteria formulated in the 
requirements for assessment panels 7> and the accredita- 
tion frameworks. In Flanders panels are approved bythe 
Recognition committee and in the near future bythe 
evaluation body VLUHR a> after a formal advise by NVAO, 

7> Guideline 'Requirements for the composition of panels' of August 201 1, 
[Leidraad Eisen aan de panelsamenstelling, Augustus 2011]. 

8> Manual 'Site visits VLIR and VLHORA', [Handleiding onderwijsvisitaties VLIR 
en VLHORA, Brussel, September 2008]. Since 2011 the evaluation bodies of 
VLIR and VLHORA merged into VLUHR; the only quality assurance agency in 
Flemish higher education. VLIR and VLHORA are stiü used separately to repre- 
sent respectively the universities and the university colleges in Flanders. 



using the protocol of VLUHR. A panel has to be approved 
before the assessment can take place. 

> Training of secretaries to assessment panels in the 
Netherlands 9> by NVAO. In Flanders the secretaries will 
be trained by VLUHR in consultation with NVAO. 

> A clear procedure for the review and assessment ofstudents' 
final assignments w> is used bythe assessment panels in 
the Netherlands. Acomparable procedure, included in the 
mentioned protocol of VLUHR, is applied in Flanders. 

> The application ofdomain (cluster) assessments in the entire 
higher education sector in the Netherlands and Flanders. 
Recently, the ministry of education has ruled that the Dutch 
professional higher education sector should also adopt the 
cluster approach in their assessments. 

After completion of the assessment report and acceptance 
bythe institution, 11> this can be submitted bythe institution 
to the NVAO, together with an application for accreditation. 
NVAO has put in place several measures to ensure the 
consistency of the evaluation of assessment reports. 

> The use of detailed manuals for the evaluation of reports, 
which are updated on a regular basis. 

> Frequent staff meetings where issues regardingthe 
evaluation of assessment reports are discussed. 

> Each application is evaluated by a policy advisor and a 
responsible board member. If the evaluation necessitates 
this, the NVAO asks other staff members or board members 
for a second opinion. A final decision on an application is 
taken by all board members in a joint session on the basis 
of a detailed propsosal. 

> The applicant institution may be asked to provide additional 
information.to ensure that each application isfully 
documented according to the criteria outlined in the 
frameworks. Sometimes NVAO applies the instrument of 
additional assessment ('Verificatie' in the Netherlands, 
'Aanvullende beoordeling' in Flanders). 

Self-evaluation 

The new frameworks contain less standards and criteria than 
the previous frameworks. In the evaluation of the accredita- 
tion system in 2008 all stakeholders mentioned the use of 

9> NVAO trained about 1 85 secretaries in the Netherlands in 201 1 and 201 2. 
After successful completion of the training a secretary is certified for one 
year and his performance is monitored by NVAO based on the panel 
reports he produces. VLUHR trains the secretaries in Flanders as they are 
usually employees of this organisation. 

I 0> 'NVAO guideline for the purpose of the assessment of graduation projects 

during site visits' of July 201 1 [Richtlijn NVAO beoordeling afstudeerwerken 
door panels tijdens het visitatieproces voor accreditatie en (indien van 
toepassing) een toets nieuwe opleiding, 8Juli2011]. 

I I > In Flanders the assessment report is published on the website of VLUHR 

before the institution applies for accreditation by NVAO. 
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open f rameworks as a strong element. Open frameworks 
enable programmes and institutions to demonstrate required 
quality in their own specific way and allow for profiling and 
differentiation. 

It is the opinion of NVAO that the accreditation system has 
improved in the second phase, especially regarding the 
consistent application of criteria. The introduction of the 
recovery period, the training of secretaries and clear 
procedures for the assessment of students' assignments 
have reinforced the system. 

Another important improvement is the appointment of panels. 
In the near future panels in the Netherlands will be appointed 
and reimbursed by NVAO. The institutions will then reimburse 
NVAO. NVAO considers this change as a meaningful reinforce- 
ment of the independency and authority of panels. In the 
previous accreditation system the quality assurance agencies 
operated on request of the institutions and this relationship 
has been indicated as a weak point in the system. 
The appointment of panels in Flanders is still executed by 
VLUHR and the recognition committee. AU stakeholders agree 
on this procedure, partly because the former QAA's from VLIR 
and VLHORA are compliant with the ESG from ENQA and are 
EQAR-registered. In the new accreditation framework VLUHR 
will consult NVAO on the composition of panels before taking 
a decision. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> Open frameworks contain less detailed criteria and its proper 
application depends for a major part on the expertise of panels 
and the way they apply the criteria. NVAO should monitor 
whether panels apply the criteria in a consistent manner. 

> The application of the four-point scale should be monitored 
thoroughly in order to achieve consistent use and a 
well-balanced score motivation bythe panels and NVAO. 

3.4/ Processes fit for purpose 

AU external quality assurance processes should be designed 
specifically to ensure their fitness to achieve the aims and 
objectives set for them. 

Facts and findings 

The Netherlands decided on putting in place an external 
quality assurance system (EQA) based on peer and expert 
review and open frameworks as the best system to achieve 
three main objectives of EQA: 



> To guarantee that all accredited programmes have the 
required quality; 

> To assess the overall quality of a study programme on a 
four-point scale; 

> To contribute to the enhancement, profiling and differentia- 
tion of higher education programmes. 

As mentioned, in Flanders the new accreditation system is 
still under construction. Both in the current and the planned 
system peer and expert review form the core of the system, 
based on open standards. In Flanders the judgement on the 
standards will be verified in technical annexes. It is not 
decided yet whetherthe four-point scale will be applied on 
the study programme level, next to each Standard. 

In the (initial) accreditation procedure roughly three steps can 
be distinguished. 

> The writing and submission of the self-evaluation report 
(in the case of accreditation) or the information file (in the 
case of initial accreditation) by the institution. The writing of 
the self-evaluation report is guided bythe protocols of the 
QAA's, which are based upon the NVAO frameworks. 

> The composition of the assessment panel, the actual site- 
visit and the assessment by the panel. A panel consists of at 
least four experts including a student member.The expertise 
of the panel should include knowledge of the discipline, 

of the professional field, of education in the domain, of the 
international dimension, and experience in reviewing or 
auditing. 

Student members are trained by NVAO for the procedures 
for initial accreditation and institutional audit. Panel 
members participating in the accreditation procedure are 
instructed bythe quality assurance agencies 

> The third step is the decision-making process by NVAO. 
If NVAO takes a negative accreditation decision, the 
programme may be granted a recovery period orthe 
application may be withdrawn by the institution. 

The voluntary institutional audit is a new and second element 
in the new accreditation system. It comprises an assessment 
by an audit committee of whether the institution is 'in control' 
of the quality and the achieved learning outcomes of its 
programmes. In order to answer this question the audit 
committee assesses the adequacy of the institutional 
strategy, its implementation through policy and the educa- 
tional processes on institutional level. Institutions that pass 
the institutional audit successfully are entitled to submit a 
limited programme assessment (LPA). Institutions that don't 
opt for an institutional audit or have failed the audit submit an 
extensive programme assessment (EPA). Flanders still has to 
decide about some elements of this system, but it is clear 
already that the institutional audit and LPA will be compulsory. 
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An important objective of the new accreditation system is the 
reduction of the bureaucratie burden on institutions caused 
by the (programme) assessments.The application of IA and 
LPA is supposed to save time and energy in comparison with 
EPA. However, in the starting phase of the new system the 
combined process of institutional audit and programme 
assessments might cause additional work.The NVAO has 
committed itself to a reduction of 25% of the administrative 
burden of programme accreditation. This was an important 
condition for the stakeholders to accept the new system. For 
this reason NVAO will continue to monitor the bureaucratie 
loads the new accreditation system entails. It can do so by 
limiting the extent of the self-evaluation report following the 
decrease of the numbers of standards, by negotiating with the 
QAA's on shorteningthe duration of site-visits specified in 
their protocols, and by making sure that the reports are clear, 
self-evident and to the point, so that additional requests for 
clarification or additions during the accreditation process can 
be prevented or limited. 

A new element of the accreditation system in the Netherlands 
is the wider application of distinctive features. 
Now institutions can apply for distinctive features at 
programme and institutional level in order to enlarge the 
possibilities of profiling and differentiation, which is a broadly 
accepted objective in Dutch higher education. . 
Up till 1 January 201 2 NVAO awarded 23 distinctive features 
and 1 9 applications for distinctive quality (the score Excellent 
on a Standard of the framework) in the Netherlands. 
In Flanders the distinctive quality feature is part of the system 
as well. Up till 1 January 201 2 NVAO awarded 1 3 applications 
for a distinctive quality feature in Flanders. 

NVAO carries out the procedure for initial accreditation 
without involving a QAA in the Netherlands and in Flanders, 
including the evaluation of the assessment process. 
Evaluation results show that institutions and panels are 
quite satisfied with the process. However, institutions 
indicate that deadlines should be respected better. 

Self-evaluation 

The current accreditation system heralds a new approach. 
The previous system focused on establishing basic quality of 
programmes as required for accreditation, whereas the new 
system challenges programmes and institutions to set higher 
targets. New elements were introduced in Dutch law regarding 
differentiation and profiling. NVAO is in favour of these 
changes as they challenge institutions and programmes to 
improve quality by defining a more explicit profile. 



The new accreditation system operates in the Netherlands 
since 1 January 201 1.The first impressions and experiences 
are favourable.The institutional audit causes staff to reflect 
more intensively on internal quality assurance at an institu- 
tional level. The assessed institutions considerthe audit a 
valuable learning experience that will positively affect the 
quality of the programmes they offer. 

As a result of the introduction of LPA and EPA the first panel 
reports prove to be more informative and transparent than 
previous reports, and enable NVAO to lessen its workload 
duringthe decision making on programme accreditations. 

The current system in Flanders operates to the considerable 
satisfaction of the stakeholders. VLUHR has an important role 
in the system as it organises the programme assessments 
and the composition of the panels. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> The implementation of the new frameworks impliesthat 
institutions and programmes have to get used to the new 
information files and self-evaluation reports. Therefore it 
may take longer beforethe 25% reduction of administrative 
burden will be achieved. All stakeholders should be aware of 
that. 

> During the coming years, NVAO will have to carefully monitor 
the composition and the operation of panels, as well as the 
operation of secretaries, as the success of system depends 
for a large part on the quality on their work. 

> The training of panel members. NVAO trains the chairper- 
sons of panels of the institutional audit, butthe panels 
members themselves are not trained by NVAO. Quality 
assurance agencies instruct panel members precedingthe 
review. NVAO should consider whether a training for panel 
members is a realistic objective. 

> In Flanders, the cooperation with the VLUHR concerning the 
quality of panels and reports should be intensified, as it is 
subject of a mid-term evaluation of the new accreditation 
system in 2017-2018. 

3.5/ Reporting 

Reports should be published and should be written in a style 
which is clear and readily accessible to its intended readership. 
Any decisions, commendations or recommendations contained 
in reports should be easy for a reader to find. 
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Facts and findings 

The assessment reports and decision documents have a fixed 
and structured format to increase readability. In the last two 
years NVAO trained secretaries in NL (partly employees of the 
quality assurance agencies) in report writing. Special atten- 
tion has been given to the newly introduced executive 
summary of the report. The new summary has to be very 
concise in order to inform an interested reader at a glance 
about the most relevant characteristics of the programme. 
Additionally, every report contains a scoring table and a sepa- 
rate paragraph with recommendations.The summary and the 
scoring table are integral part of the decision document. 
Regardingthe readability of reports NVAO started in 201 2 an 
evaluation project to investigate whetherthe new reports 
respond wellto the needs of the intended readers. 

Both NVAO's decisions and the panel assessment reports are 
made public on NVAO's website (www.nvao.net). The site is 
equipped with a search tooi to find information about a specific 
programme or an institution. It is NVAO's aim that relevant 
information in the assessment reports and decisions should 
be easily accessibleforthe intended readers, such as students, 
their parents, the professional field and.finally, the general 
public and the government. In this regard NVAO is considering 
the use of social media to inform its main stakeholders more 
adequately about decisions and reports. The distribution of 
specific website links to target groups is an option. 

On a regular basis NVAO gives feedback to the quality 
assurance agencies and secretaries concerningthe quality of 
the reports they produced. In Flanders secretaries receive 
feedback from VLUHR in cooperation with NVAO. On 1 4 May 
201 2, for instance, an evaluation meeting between secretaries 
ofVLUHRand NVAO staffis planned regardingthe quality of 
reports. The new accreditation system in Flanders gives 
NVAO the authority to reject a report, when it is found lacking 
sufficiënt quality to serve as a basis for an accreditation 
decision. The new system also requires a readable summary 
to be included in the report. Both options already exist in the 
new accreditation system in the Netherlands. 

Additional formal information about accredited programmes 
and higher education institutions is provided by the registers 
on higher education in the Netherlands and Flanders: 
respectively CROHO 1 2> and H0R 13> . 

1 2> Central register for higher education programmes in the Netherlands 
[CROHO: Het Centraal Register Opleidingen Hoger onderwijs]. CROHO is 
maintained by the governmental organisation DUO, [Dienst Uitvoering 
Onderwijs in Groningen]. CROHO is accessible for the public (www.croho.nl). 

13> Higher Education in Flanders ]H0R: Het Hogeronderwijsregister] HOR is 
developed and maintained by NVAO and is accessible for the general 
public (www.hogeronderwijsregister.be). 



Self-evaluation 

NVAO monitored the quality of reports during the previous 
accreditation period. With the introduction of the new system 
in the Netherlands and Flanders NVAO agreed with the quality 
assurance agencies to improve the readability of the reports. 
The first results in the Netherlands are promising. It is 
expected that the main findings and conclusions of a report 
will be read better by students, employers and other involved 
groups. 

NVAO's website contains all accredited programmes in the 
Netherlands and Flanders, but the site does not reach enough 
visitors. It is the opinion of NVAO that especially (future) 
students, their parents and employers can profit from 
adequate information about the quality of study programmes. 
This information can help future students to choose a study 
programme and employers, for instance, in case of deciding to 
engage themselves in a partnership with an institution of 
higher education. 

As it appears to be difficult to inform stakeholders adequately, 
NVAO will explore other options to communicate effectively 
with these groups, for instance through social media. At the 
moment NVAO acts within its ability to inform especially 
students and employers. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> NVAO should monitor whether the readability of the 
assessment reports and NVAO's decisions is increasingly 
appreciated by students, employers and other involved 
groups. 

3.6/ Follow-up procedures 

Quality assurance processes which contain recommendations 
for action or which require a subsequent action plan, 
should have a predetermined follow-up procedure which is 
implemented consistently. 

Facts and findings 

In the previous accreditation system in the Netherlands 
yes/no decisions were made unconditionally.Therefore, 
the NVAO could not formally demand improvement actions. 
The higher education institutions were responsible for the 
follow-up on the recommendations of the assessment panel, 
and the improvements were subject to assessment in the 
subsequent evaluation procedure. 

Because of the lack of a recovery period in the first phase in 
the Netherlands NVAO decided to introducé the instrument 
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specific agreements in order to create an appropriate arrange- 
ment regardingthe accreditation of particular programmes. 
When accreditation cannot be refused to a programme, but 
serious doubts remain, NVAO can propose to the board of the 
institution to deliver a report of an additionat panel assess- 
ment after three years.The agreement is voluntary, but in 
almost all cases institutions have agreed to this proposal in 
recent years. 

In Flandersthe instrument of specific agreements has not 
been applied, because of the more formal approach regarding 
accreditation outlined in Flemish legislation and the existence 
of a recovery period from the start of the system in 2005. In 
the current system a recovery period is granted by the Flemish 
government. In the new Flemish system NVAO will decide on 
this period with a maximum of three years. In the current as 
well as in the future system in Flanders the new institutional 
audit and the programme accreditation decision focuses 
specifically on the follow-up based on the recovery plan. 

An official follow-up procedure, as meant here in the ESG 
guideline, is not part of the new accreditation system either, 
but two measures introducé elements that provide equivalent 
functionality toa follow-up procedure in the new system in 
the Netherlands: 14> 

> The implementation of the recovery period in the procedure 
of programme accreditation. 

> The authority of NVAO to take a conditionally positive 
decision in the procedure of initial accreditation and 
institutional audit. 

In case of the application of a recovery period the programme 
has to present a convincing recovery plan before the recovery 
period is granted. At the end of the recovery period the 
achieved improvements haveto beassessed positively by an 
assessment panel before accreditation can be granted. 
In case of a conditional decision in the Netherlands a NVAO 
panel must assess whether the programme or the institution 
meets the conditions within the set conditional period. 
No follow-up procedure is applied when a programme 
is accredited on the basis of a report that contains recom- 
mendations from the assessment panel. In that case, the 
follow-up on these recommendations will be assessed in 
the subsequent accreditation procedure (after six or eight 
years [NL-FL]). 

A list of recommendations of the kind mentioned above can be 
included in the panel report in the new system, and provides 
an additional incentive for a good follow-up by the programme 
management or the institution. 

1 4> A recovery period has been part of the accreditation system in Flanders 
since the start in 2005. 



The new situation in The Netherlands and Flanders can be 
illustrated as follows for each of the assessment procedures: 

Formal 

Institutional audit: in case of a conditional decision 
> a follow-up procedure is applied 

Accreditation: decision: 

□ positive: > recommendations are part of the panel report 

□ negative: > recovery period > follow-up procedure is applied 

> no recovery period > shutdown programme 

Initial Accreditation: decision 

□ conditional for 1 year > follow-up procedure is applied 

□ unconditional > FL: approval for 'the length andtwo ns> 

> NL: variable period to decide by NVAO 

Informal 

Specific agreements (NL) 

Self-evaluation 

During the previous accreditation system NVAO has repeat- 
edly stated that a recovery period in the Netherlands would 
improve the Dutch system. A simple yes/no-decision seems 
clear and strong at first sight, but it invites anticipatory 
behaviour of programme representatives and panels to avoid 
negative outcomes.Therefore opponents of external quality 
assurance mention 'window dressing' by programme 
representatives as significant disruptive behaviour during 
site-visits. On the other hand advocates of the system 
emphasize the peer review approach applied in an environ- 
ment of trust and quality culture as the best condition to 
assess educational programmes and to contribute to its 
enhancement. 

NVAO is convinced that a recovery period and systematic 
follow-up on the basis of a recovery plan contribute to the 
flexibility of the system, to the independent and expert's 
judgement of the panels and to an open attitude of 
programme representatives. Therefore NVAO welcomes the 
recovery period (NL) and the possibility of conditional 
decisions regarding initial accreditation (NL) and the keeping 
of the recovery period in FL. 

Judgement 

NVAO complies with the Standard. 



1 5> in Flanders initial accreditation is given for the length of the programme 
and two years (e.g. a professional bachelor programme of three years is 
approved for five years). 
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3.7/ Periodic reviews 

External quality assurance of institutions and/or programmes 
should be undertaken on a cyctical basis. The length of the 
cycle and the review procedures to be used should be clearly 
defined and published in advance. 

Facts and findings 

The accreditation period of programmes in the Netherlands 
is six years, in Flanders it is eight years. After an initial 
accreditation, reassessment of the programme and reaccredi- 
tation should be finalized within six years (in the Netherlands) 
and within the length of the programme and two years (in 
Flanders). These cycles are registered in Dutch and Flemish 
legislation. Within a set period before the accreditation period 
has expired, the institution should submit an assessment 
report and an application for accreditation to NVAO. 
The recommendation of the former ENQA committee in 2007 
to equalize the accreditation cycles in the Netherlands and 
Flanders has not yet been implemented. In the opinion of 
NVAO a cycle of eight years is too long and that is why NVAO 
discussed this point with relevant stakeholders. However, 
only the governments of Flanders and the Netherlands have 
the authority to change the length of the cycles. 

The six-year re-assessment timeframe for accreditations is 
considered to be adequate. However, the six-year re-assess- 
ment timeframe for initial accreditation in the Netherlands is 
considered to be too long. In the new Dutch accreditation 
system NVAO can decide on the duration of the initial accredi- 
tation, with a minimum of one and a maximum of six years. 
In Flanders the duration of the initial accreditation will remain 
the length of the programme and two years, while the 
accreditation period of study programmes willfirst remain 
eight years, before being reduced to six years in order to come 
in line with the institutional reviews. 16> 

Self-evaluation 

The accreditation process is undertaken on a cyclical basis, 
which is clearly defined in both Dutch and Flemish legislation. 
AU frameworks and procedures are published in advance. 
NVAO has addressed the topic of equal accreditation cycles 
between the Netherlands and Flanders in the meetings of 
both Ministers [Comité van Ministers] in recent years. It is 
expected that the third round of accreditation in Flanders 
(probably to start in 2021) will offer an accreditation cycle of 
six years. The new authority of NVAO in the Netherlands to 
decide on the duration of an initial accreditation is an impor- 
tant improvement, as is the shift of authority f rom the Flemish 

1 6> Flanders uses the term Institutional review instead of Institutional audit. 



government to the NVAO to decide on the recovery period of a 
study programme in Flanders, before this can be accredited. 

Judgement 

NVAO complies with the Standard. 

3.8/ System-wide analyses 

Quality assurance agencies should produce from time to time 
summary reports describing and analyzing the genera/ findings 
of their reviews, evaluations, assessments etc. 

Facts and findings 

In chapter 9.1 the topic of comparative analyses is discussed 
in the context of the follow-up on the recommendations of the 
ENQA review committee in 2007. In the past years NVAO 
completed eight comparative analyses of various kinds. 
Attachment 1 provides a full description of these completed 
analyses, in which a distinction can be made between 
comparative analyses on the level of a cluster of the same 
study programmes and at the level of a domain. 
In 201 2 NVAO will start an international benchmarking of four 
professional bachelor programmes in Switzerland, Finland, 
Flanders and the Netherlands in order to compare accepted 
quality indicators in these countries. 

Until recently, initiatives to comparative analyses were based 
on facts and findings in assessment reports or on requests 
from the higher education sector or the Minister. 

Cluster assessment contributes to comparative analyses. 
Nearly all Dutch and Flemish programmes will be subject to 
clustered assessments organised by QAA's, since it has been 
decided to reintroduce this approach in Dutch professional 
education. The reports of the clustered programmes are 
published in the same period accompanied by a comprehen- 
sive panel report containing general considerations regarding 
the involved discipline and programmes. This report provides 
an additional, more generic view on the performance of 
these programmes, which is also used by NVAO for its own 
comparative analyses. 

Besides comparative analyses on the level of clusters or 
domains, system-wide analyses on the level of the system 
itself are internally planned, as part of NVAO's internal quality 
assurance (see chapter 4.8) or as part of piloting the new 
accreditation system or its implementation. Both in the 
Netherlands and Flanders system-wide evaluations of the 
new accreditation system are foreseen, which will be (partly) 
done by NVAO. 
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As NVAO's new strategie policy underlines the importance 
of comparative and system-wide analyses NVAO recruited a 
new staff member assigned to do quantitative and qualitative 
research to underpin the analyses. 

Self-evaluation 

Despite the substantial effort of NVAO regarding comparative 
analyses, NVAO did not have a clear policy on this theme and 
time in recent years. In the new Strategie Policy Statement 
of NVAO (201 2 - 201 6) the NVAO task of system-wide and 
comparative analyses is emphasized and should result in a 
clear agenda on this topic for the coming years. 

Another point for improvement concerns the follow-up of the 
results of system-wide or comparative analyses. The analysis 
only makes sense if the results are thoroughly discussed with 
representatives of the institutions and programmes involved. 
Subsequently, the institutions should accept the results as 
meaningfulforfurther improvement of the programmes 
involved. NVAO should take more initiatives to start these 
discussions while and after completing a system-wide or 
comparative analysis. 

In conclusion it can be stated that NVAO completed eight 
comparative analyses in the past years and is therefore 
compliant with the guideline of this Standard, as it is required 
'that summary reports must be produced from time to time'. 
Nevertheless, NVAO has to work on implementing its new 
policy regarding system-wide and comparative analyses. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> To implement a clear policy and activities regarding system- 
wide analyses through yearly planning. 
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Standards 



and 



Guidelines 
part 3 



4.1 / Use of external quality assurance 
procedures for higher education 

The external quality assurance ofagencies should take into 
account the presence and effectiveness of the external quality 
assurance processes described in Part 2 of the European 
Standards and Guidelines. 

In chapter 3 it is described how NVAO takes into account all 
European Standards and Guidelines of Part 2. 

Judgement 

NVAO complies with the Standard. 

4.2/ Official status 

Agencies should be formally recognised by competent public 
authorities in the European Higher Education Area as agencies 
with responsibilities for external quality assurance and should 
have an established legal basis. They should comply with any 
requirements of the legislative jurisdictions within which they 
operate. 

Facts and findings 

NVAO is the official, public and binational accreditation 
organisation in higher education in the Netherlands and 
Flanders. lts structure and duties are described in the 
binational Treaty 17> and in both national legislations. 
NVAO reports to the Dutch and Flemish Parliaments via 
their Ministers of (Higher) Education. NVAO's annual report 
is used for accountability purposes. 

NVAO was granted the status of an autonomous administra- 
tive body with legal rights accordingto Dutch legislation. 
Consequently, NVAO does not resort under a particular 
minister and is not subject to ministerial responsibility. 
NVAO has full decision-making powers concerning decisions 
about (initial) accreditation. 

However, NVAO is accountable to the Committee of Ministers, 
which approves its budget, the annual report and the annual 
accounts. In accordance with theTreaty, the Committee of 
Ministers can only intervene in case of serious neglect by NVAO 
of its (initial) accreditation task.threateningthe execution of 



1 7> Treaty between the Kingdom of the Netherlands and the Flemish Commu- 
nity of Belgium regarding the accreditation of programmes within Dutch 
and Flemish higher education, The Hague, 3 September 2003 [Verdrag 
tussen het Koninkrijk der Nederlanden en de Vlaamse Gemeenschap van 
België inzake de accreditatie van opleidingen binnen het Nederlandse en 
Vlaamse hoger onderwijs]. Referred to in this document as the Treaty. 



that task.The Committee of Ministers can thus only intervene 
in the functioning of NVAO, but not in NVAO's decision-making. 

Self-evaluation 

The official status offers NVAO a good formal structure to 
guarantee its independence. lts binational nature is an 
unique and interesting example regarding the objective of the 
Bologna treaty of 1 999 to expand cross border cooperation 
and to achieve one European higher education area. During 
the last years lots of international delegations visited NVAO, 
because of its binational nature and to learn from the 
cooperation in the field of external quality assurance between 
the Netherlands and Flanders. 

From the start of NVAO in 2005 it was expected that a 
binational organisation would reinforce the Dutch-Flemish 
cooperation in EQAHE, which also came about. It resulted for 
instance in a frequent exchange of peers in the panels 
between the Netherlands and Flanders, and NVAO applies 
the same criteria to decide on Dutch and Flemish applica- 
tions. However, progress on the mutual recognition of degrees 
between the two countries has been limited.The planned new 
Treaty offers a new opportunity in this case. The making of one 
unified list of (initially) accredited programmes to underpin 
mutual recognition would be a step forward. Even a generic 
recognition of the levels of the degrees is in sight. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> From its binational nature NVAO contributes to the align- 
ment of the Dutch and Flemish systems of higher education, 
includingthe mutual recognition of degrees. Little progress 
has been made in recent years in this respect and therefore 
more attention must be paid to it. A generic recognition is in 
sight thanks to the attention drawn by NVAO and others. 

4.3/ Activities 

Agencies should undertake external quality assurance activi- 
ties (at the institutional or programme level) on a regular basis. 

Facts and findings 

The tasks of NVAO are described in theTreaty. They are further 
stipulated in the Netherlands in the Dutch Act 18> on higher 
education and research and can be summarized as the 
(initial) accreditation of programmes of institutions of 

1 8> [Wet op het hoger onderwijs en wetenschappelijk onderzoek, 1 januari 
2011], changed version due to the new accreditation system. 
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higher education and the assessment of these institutions 
(Institutional audit). 

In Flanders, the Flemish Act of 4April 2003 19> formsthe 
legal basis for (initial) accreditation in higher education and 
stipulates that the responsibility for (initial) accreditation of 
programmes is assigned to NVAO.The number and type of 
NVAO's decisions up to 1 January 201 2 are illustrated in 
chapter8.1,table7. 

Another important task of NVAO stipulated by theTreaty 
concerns the international perspective on the decisions and 
position of NVAO.This aspect is dealt with in chapter 8.4: 
The international assignment of NVAO. 
Since 30 April 2009 NVAO has also been empowered to 
validate the discipline-specific learning outcomes of 
bachelor's and master's programmes in Flanders. 

The structures of the external quality assurance systems in 
the Netherlands and Flanders will be discussed in Chapter 7. 
AU higher education programmes in the Netherlands and 
Flanders must be accredited on a regular basis; in the 
Netherlands every six years, in Flanders every eight years. 
The Dutch institutions in higher education are subject to 
an institutional audit every six years. In the new system in 
Flanders institutions probably will be subject to an institu- 
tional audit every six years. 

NVAO is charged with several additional tasks, which are 
described in detail in chapter 8.5 and attachment 2. 
These additional assessment tasks enable NVAO to acquire 
a better insight in quality issues in Dutch and Flemish higher 
education and internationally. 

An example of the organisation of NVAO's processes of (initial) 
accreditation is further described in the workflow scheme in 
attachment 3. 

Self-evaluation 

In evaluation discussions aboutthe accreditation process 
with institutions in higher education and with quality 
assurance agencies in the Netherlands, the major comments 
of both stakeholders concern the following points: 

> NVAOshould respect the formal deadlines better. 

> additional questions and additional assessments by NVAO; 
stakeholders believe that these additional activities unnec- 
essarily increase the administrative burden in the system. 

In 201 1, parallel to the regular meetings with VLUHR and the 
Flemish cabinet, NVAO visited in Flanders all higher education 

1 9> Decree on the restructuring of higher education in Flanders, 4 april 2003. 



institutions to discuss their experiences with application 
procedures for accreditation at NVAO. In general NVAO 
respects deadlines in Flanders due to the strict regulations 
in Flanders. Regarding additional questions and additional 
assessments the Flemish higher education institutions have 
the same opinion as the Dutch institutions. The results of 
these evaluation meetings in both countries are processed 
in NVAO's IQA activities. 

As there are approximately 600 existing programmes and 
1 00 new programmes to be accredited in the Netherlands and 
Flanders each year, the (administrative) assessment burden is 
high. One of the aims of the new system is to lower this burden 
and to focus on the heart of the matter: 'does the programme 
achieve what it preaches on an internationally accepted 
level?' NVAO expects that better and convincing reports will 
be a positive outcome of the new assessment process and, 
as a consequence, that NVAO can reduce its additional 
assessment activities. This would be an improvement of the 
accreditation process. 

NVAO played a proactive role in the period 2008-201 in the 
evaluation and development of the new accreditation system 
and, naturally, supports it strongly. NVAO is convinced that the 
new accreditation system in the Netherlands challenges 
institutions to work on objectives as profiling, differentiation 
and quality enhancement. The new accreditation system in 
Flanders will probably incorporate the objectives differentia- 
tion and quality enhancement. Profiling is not an actual issue 
in Flemish higher education. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> NVAOshould respect deadlines better. A newly appointed 
coördinator for assessments manages the workflow in the 
organisational process, which should contribute to a timely 
completion of applications. A careful and timely completion 
of primary tasks must have priority. Additional tasks should 
be part of a structured and realistic yearly plan. 

> NVAOshould monitor whether the aim of decreasing 
numbers of additional assessments and additional ques- 
tions will be achieved in the coming years. It is obvious that 
from the outset NVAO should try to avoid duplicating the 
work of panels. 
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4.4/ Resources 

Agencies shoutd have adequate and proportional resources, 
both human and financial, to enable them to organize and run 
their external quality assurance process(es) in an effective and 
efficiënt manner, with appropriate provision forthe develop- 
ment of their processes and procedures. 

Facts and findings 

NVAO has a (General) Board of 1 1 members 20> .The chair and 
three other Board members constitute the Executive Board. 
NVAO also has an Advisory Council of 1 4 members 21 '. 

The workforce of NVAO consists of 56 people (53,1 fte): four 
executive Board members, one managing director, 30 policy 
advisors, one Dutch and one Flemish legal advisor 22 ' (together 
29,1 fte 23 ') and 20 other (partly supporting) staff (1 9 fte). 
Four policy advisors are available for international assign- 
ments.The staff includes policy, legal and communication 
advisors and supporting staff (policy secretariat.finances 
and personnel, records department and general services). 

Decisions are prepared by the policy and legal advisors and at 
least one Executive Board member. In 201 the Board (re) 
appointed four policy advisors as coördinators forthe areas 
Assessments, New accreditation system, Internal quality 
assurance and Internationalisation. In their role as coördina- 
tors policy advisors remain accountable to the director. 
See the organisational chart (attachment 4). 

Core competences of NVAO's staff 
AU academie staff hold a master's degree, three hold a PhD 
degree.The executive board of NVAO consists of four 
members, two from the Netherlands, two from Flanders. 
The primary responsibility of the board is the accreditation 
process and the accreditation decisions. A managing director 
is responsible for the management process of NVAO. 

The acquired professional experience of NVAO's staff is varied. 
AU board members and the director have a longer career in 
(higher) education in board or management positions. 
Most staff acquired substantial work experience in higher 
education as teacher, developer, manager, researcher, 
inspector for higher education or policy advisor, before being 
employed at NVAO. A small group of junior staff was recruited 

20> The members of the General Board act on personal name and don't 

represent an organisation. 
2 1 > The members of the Advisory Board represent stakeholder organisations. 
22> The activities of the Flemish legal advisor are outsourced by NVAO to a 

Flemish legal consultancy firm. 
23> About one-third of the policy advisors is Flemish, two-third is Dutch. 



to fulfil a kind of traineeship before beingfully employable 
in NVAO's work processes. 

Compared to similar members within the ENQA network, NVAO 
staff is on par 24> . NVAO has built a staff group of good quality. 

Finances 

NVAO has an annual budget of approximately € 6 million, 
which is financed jointly by the Netherlands (60%) and 
Flanders (40%). Each year NVAO draws up a budget which 
is then decided upon by the Committee of Ministers and both 
Parliaments. In the Netherlands and Flanders NVAO applies 
fixed rates by law.The rate for an application initial accredita- 
tion is maximum € 1 5.000 in the Netherlands and € 5.000 in 
Flanders. For an application accreditation the rate is € 750 in 
the Netherlands and € 500 in Flanders. In case of additional 
tasks and assessment assignments abroad NVAO charges 
cost-coveringfees. 

Facilities 

Since 2005 NVAO is located at the Parkstraat in The Hague, 
nearthe city and the Dutch parliament and the Dutch ministry. 
Good facilities are available here. Several meeting rooms 
for smaller and larger groups are available to organise 
conferences, seminars and meetings. Presentation and 
IT-facilities are up-to-date. In November 201 1 a new informa- 
tion system was introduced to optimize the work-flow. 25> 

Self-evaluation 

NVAO has been funded sufficiently by both governments for 
its primary tasks in recent years. However, due to the 
substantial size of additional tasks that were assigned to 
NVAO, additional funding was needed. NVAO agreed with both 
ministries in the Netherlands and Flanders that additional 
tasks will be organised and budgeted on a project basis. 
NVAO informs the ministries in advance about the expected 
costs. The latter can decide whether to continue the project 
initiative or not. 

AU public organisations, including NVAO are currently subject 
to public cuts initiated by the governments in the Netherlands 
and Flanders. These cuts don't affect NVAO seriously up to now. 

Duringthe years NVAO developed a good quality of staff with 
varied competences and work experiences. Dutch and Flemish 

24> In 201 1 the ENQA network Internal Quality Assurance completed a first 
benchmark study on Staff development among 5 members of ENQA 
[QAA (UK) , ACQUIN (GE), EVA (DK), ANECA (SP), NVAO (NL)]. NVAO 
participated actively in this benchmark. The results will be presented 
on the ENQA website in Spring 201 2. 

25> NVAO uses a document management system (DMS) of RICOH. 
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board and staff members form a productive mixture and the 
chemistry between the Dutch and the Flemish part of NVAO 
is good. Flemish staff are deployed on Dutch and Flemish 
applications and vice versa. Perceived culture differences 
are rather stimulating and don't cause problems. A point for 
improvement is the further development of expertise of staff 
on specific themes in higher education. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> NVAO should invest in further development of specific, 
theme-oriented, higher education expertise of staff. 

4.5/ Mission statement 

Agencies should have clear and explicit goals and objectives 
for their work, contained in a publicly available statement. 

Facts and findings 

In 201 2 NVAO renewed its Strategie Policy Statement 26 " for 
the period 201 2-201 6, which includes NVAO's new mission, 
that was approved by the General Board on 1 9th March 201 2 
and is published on NVAO's website and included in this report 
as attachment 5. 

Mission 

NVAO has defined its new mission as follows: 

NVAO is the independent and authoritative binational 
accreditation organisation set up by the Flemish and Dutch 
governments, whose primary goal it is to provide an expert and 
objective judgement of the quality of higher education in Flanders 
and the Netherlands. NVAO does this with a constructive, 
critical attitude, respectingthe autonomy of institutions and 
their primary responsibility for the quality of their education, 
and with an open eye for the growing international context. 
NVAO is open, clear and transparent towards society and all 
concerned, especiallythe institutions of higher education and 
the students. 

In its Strategie Policy Statement NVAO states that it considers 
institutions primarily responsible for quality assurance and 
quality improvement. NVAO is assigned to assure that 
programmes meet the required standards and to stimulate 
the quality debate, giving account of its procedures, 
disseminating'good practices' and visiting the institutions 

26> Strategie Policy statement NVAO 2012 - 2016, 13 February 2012 
[Strategische beleidsverklaring NVAO 2012-2016] 



and programmes. NVAO respects the autonomy of institutions, 
which are responsible for their internal quality assurance 
and fortaking measures for quality improvement. 

lts mission commits NVAO to be strongly present and 
communicative in the quality debate in higher education. 
In practice, NVAO's role in stimulating debate in recent years 
can be illustrated by referringto its conferences, seminars, 
intensive meetings with the stakeholders and a large number 
of visits each yearto institutions in the Netherlands and 
Flanders. During these visits, discussions are held with the 
management of the institution on, among other things, their 
experiences with the accreditation system and the analysis 
of a series of accreditation decisions for their programmes. 
The meetings are initiated by NVAO. In addition, members of 
the NVAO Executive Board or staff often give presentations at 
conferences. NVAO organises its own conference every two 
years either in the Netherlands or in Flanders. Furthermore, 
the NVAO newsletters, published every two months, give more 
information on several issues regarding accreditation. 

Self-evaluation 

The discussions preparing a new mission statement 
concentrated for a part on the question whether NVAO should 
concentrate on its assurance assignment, its enhancement 
assignment or both. These discussions were affected by the 
introduction of a new accreditation system, the international 
developments and recent quality incidents and their 
consequences in Dutch professional education. These events 
pushed the Dutch in the direction of stressing the assurance 
assignment, while the objective of the new accreditation 
system and the role of NVAO initially was more oriented 
towards the enhancement assignment. 

The newly introduced institutional audits, focusing on 
policies and the IQA system of the institution provide a good 
opportunity to focus on quality enhancement, like the 
planned, extended realisation of system-wide or comparative 
analyses. The latter two focus on quality enhancement rather 
than on quality assurance. NVAO is of the opinion that in a 
well-operating accreditation system the emphasis should 
be on the enhancement objective. In the final Strategie 
Policy Statement this discussion was concluded. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> NVAO should assess regularly whether it achieves a 
consistent balance between its accountability and 
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enhancement assignment, an assessmentthat should 
not by dictated by ad hoe issues in higher education. 

4.6/ Independence 

Agencies should be independent to the extent both that they 
have autonomous responsibility for their operations and that 
the conclusions and recommendations made in their reports 
cannot be influenced by third parties such as higher education 
institutions, ministries or other stakeholders. 

Facts and findings 

The independence of NVAO has been described in detail in 
chapter 4.2. Here it can be added that members of NVAO's 
Executive and General Board are mandated by the Committee 
of Ministers for four years.They are appointed on strictly 
personal capacity and not as representatives.The other 
Advisory Council members represent different stakeholders 
(see chapter 4.8). 

In order to guarantee the independence of board and staff 
members, NVAO has ruled that both cannot participate in 
applications from institutions or programmes they have 
been associated with in anyform during the last f ive years. 
Members of the Board have to be completely independent in 
taking decisions. If there is a specific application where this 
independence cannot be guaranteed, the member of the 
Board will withdraw from the decision-making process for the 
programme concerned. NVAO is responsible for recruiting its 
own staff and the decisions it takes. 

NVAO's independence is recognised by different stakeholders. 
NVAO is regarded as the quality controller of the quality 
assurance organisations' panels, site-visits and reports. 
Two examples can illustrate this. 

For instance, the Dutch parliament asked NVAO on 26 January 
201 2 to explain the operation and the first experiences of the 
new accreditation system in order to convincethe members 
of parliament of the reliability of the new accreditation system 
in the Netherlands. 

In 201 NVAO was invited by the Flemish Parliament to explain 
its work and opinions on the new accreditation system in its 
Commission of Education. In Flanders NVAO has been allowed 
as member of the Advisory group (Adviesgroep) of VLOR, the 
official Flemish Educational Council (Vlaamse Onderwijsraad) 
to advise the government about the structure and operation 
of the new accreditation system in Flanders. 
These examples illustrate that NVAO is not considered as a 
purely executive organisation bythe involved stakeholders. 
This position allows NVAO to advise independently on the 
new system. 



Self-evaluation 

From the start of the accreditation system in the Netherlands 
and Flanders (2004, NL - 2005, FL) NVAO has been completely 
independent regarding decision making. This independence is 
the best guarantee for a transparent and strong decision 
making process. 

It is not always easy to demonstrate NVAO's independence to 
all stakeholders, especially its position towards the parliament 
and the institutions. The best proof of NVAO's independence is 
that it is sometimes perceived bythe ministries and politicians 
as too soft tothe institutions, while the institutions sometimes 
blame NVAO for beingtoo severe in followingthe political 
demand for robustness and transparency. This is partly linked 
to NVAO's deliberate choice for both accreditation and quality 
enhancement (see chapter 4.5). This delicate balance can 
only be maintained as a result of NVAO's strict independence 
and its policy not to mix decision-making with advice. 
All recommendations for the improvement of programmes 
are formulated by independent panels. NVAO sometimes 
only chooses to highlight some of the recommendations. 

Judgement 

NVAO complies with the Standard. 

Point(s) of attention 

> NVAO has to communicate strongly, independently and, 
especially, proactively.Therefore a new Communications 
strategy has been set up recently. 

4.7 / External quality assurance criteria 
and processes used bythe agencies 

The processes, criteria and procedures used by agencies 
should be predefined and publicly available. These processes 
will normally be expected to include: 

> a self-assessment or equivalent procedure by the subject of 
the quality assurance process; 

> an external assessment by a group of experts, including, as 
appropriate, (a) student member(s), and site visits as decided 
by the agency; 

> publication ofa report, including any decisions, 
recommendations or other formol outcomes; 

> o follow-up procedure to review actions taken by the 
subject of the quality assurance process in the light of 
any recommendations contained in the report. 

h the previous Dutch and the current Flemish system the 
21 criteria or standards are grouped in 6 subjects (aims, 
programme, staff, facilities, internal quality assurance (IQA) 
and achieved results). These criteria were developed in coop- 



Analysis of Standards and Guidelines part 3/29 



eration with all stakeholders and are perceived as clear and 
robust, but also as burdensome for the management of 
programmes because in most institutions systems for HRM, 
IQA and other policies are often formulated and managed at 
the institutional level.Therefore, standards for accreditation 
of study programmes in the new system have been limited to 
three (general aims and intended learning outcomes, the 
tearning environment and the achieved learning outcomes), 
and an institutional audit is introduced.The standards and 
underlying criteria in the new system were developed in coop- 
eration with all stakeholders through pilots and evaluation. 
The previous standards and criteria are only maintained in the 
new system in the frameworkfor EPA.The criteria within the Dutch 
and Flemish systems are described more in detail in chapter 3.3. 

Chapter 3.4 described the successive steps of the (initial) 
accreditation procedure in the Netherlands and Flanders. 
Here follows a summary of these steps, which applies both to 
the previous Dutch and the current Flemish system, as well as 
to the new accreditation system (NL, FL in 2013). 

> The institution writes and submits the self-evaluation report 
orthe information file. 

> A panel from a quality assurance agency or an NVAO panel 
carries out a site-visit and an assessment of a programme. 

> The panel prepares an advice to the Board of NVAO and 
informs the institution about an outline of its conclusions 
of the assessment at the end of the site-visit. 

In case of programme accreditation the institution has to 
submit an application to the NVAO based on the panel report. 
In case of the new institutional audit or initial accreditation 
the NVAO panel presents the report of its findings and judge- 
ments to the board of NVAO. 

> The next step is the decision-making process by NVAO. 
If NVAO takes a negative accreditation decision on an 
application for accreditation, the institution must either 
stop the programme immediately or it can be granted a 
recovery period. If NVAO intends to take a negative decision 
on an application for initial accreditation the institution can 
withdraw the application or accept the negative decision. 

> The decision regardingthe institutional audit in the 
Netherlands can be threefold: positive, conditionally positive 
or negative. A positive decision entitles the institution to 
apply for LPA for all its programmes. A conditionally positive 
decision requires the institution to repair the deficits within 
oneyear. Anegative decision is in force for three years, in 
which period all planned assessments of programmes have 
to be executed accordingto the frameworkfor an EPA.The 
institutional audit in Flanders will not be concluded with a 
final decision; in the first round it is a learning experience, 
aiming at quality enhancement only. 



NVAO implemented several procedures and processes 
to improve the consistency of its decision making. For an 
overview, see chapter 3.3. 

Aformal follow-up procedure is not included in the accredita- 
tion system. However such a procedure has become common 
practice, as described in chapter 3.6. 

Self-evaluation 

Both the previous Dutch and current Flemish criteria were 
reviewed as compliant with the ESG in 2007. Programme 
assessment and accreditation in the new system focus on the 
content of the curricula and on the achieved learning outcomes. 
Flanders and the Netherlands are at the forefront in this 
respect, as this focus is not common in the European higher 
education area. NVAO expects that the institutional audit will 
be a strong incentive for further development of effective 
internal quality assurance on an institutional level, which will 
positively influence the quality of individual programmes. 

The new Dutch system gives NVAO additional powers which 
will improve the effectiveness of its operation. NVAO believes 
that the recovery period and the power to award a conditional 
initial accreditation are effective tools for improvement of the 
system. NVAO's authority to appoint (and reimburse) panels in 
the Netherlands and the reintroduction of the cluster assess- 
ment approach in Dutch higher education are other important 
improvements. In the new accreditation system NVAO's 
influence on the quality of panels has improved and that is 
of major importance for the quality of the system as a whole. 
In Flanders NVAO's influence on the panel composition 
has improved as well. The VLUHR will only decide on the 
composition of assessment panels after an NVAO advice. 

Judgement 

NVAO complies with the Standard. 

4.8/ Accountability procedures 

Agencies should have in place procedures for their own 
accountability. 

4.8.1 / Appeal procedures 

Facts and findings 

Both in the Netherlands and Flanders, institutions can lodge 
an appeal against (initial) accreditation decisions taken by 
NVAO. Although formal appeal procedures differ in both 
countries, the principles behind them are to a large extent 
similar.The main steps in the procedure in both countries are 
described below. 
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> Prior to taking a final decision concerning an application for 
(initial) accreditation, NVAO will inform the institution about 
its 'intended decision'. The institution then has two weeks to 
respond. Subsequently, NVAO takes its final decision. 

> An institution can lodge an internal appeal against a NVAO 
decision. To handle internal appeals, NVAO has set up an 
appeals commission that provides advice to NVAO on which 
NVAO will take its decision. 

> If the decision on an internal appeal is negative.the 
institution can lodge an external appeal against this 
decision with the Administrative Jurisdiction Department of 
the Council of State [Afdeling bestuursrechtspraak van de 
Raad van State] in the Netherlands. In Flanders, institutions 
can lodge an external appeal with the Flemish Government 
and finally with the Council of State of Belgium [Raad van 
State]. 

Since the start of the accreditation system, NVAO has received 
nine internal appeals in the Netherlands. One appeal has 
been granted. In Flanders there were seven cases of internal 
appeal up to now. One appeal was granted, two were rejected 
and four were partly granted, but did not change NVAO's final 
decision after a new assessment by a panel. 

At the time of writing, four external appeals have been lodged 
against a decision of NVAO in the Netherlands. One external 
appeal has been granted on procedural reasons.The others 
were refused. In Flanders no external appeals have been 
lodged. 

Self-evaluation 

The internal appeals procedures are set up satisfactorily. 
In the Netherlands, they have proven to function effectively. 
In Flanders, a new Procedure Regulation has been adapted in 
201 1 after an evaluation by the NVAO board and the appeals 
commission. The adaptations concern the terms of appoint- 
ment of the members of the commission, the specific 
competences of the experts in the commission and the 
decision rules concerning its advice, in which it is now clearly 
stated that the appeals commission cannot change the scores 
on its own, but only accept the appeal and advise to start a 
new assessment on the identified problematic standards. 

4.8.2/ Internal quality assurance 

Facts and findings 

From the outset, NVAO has developed a system for its internal 
quality assurance (IQA). Initially, this system had a thematic 
ratherthan a structural approach. Further development and 
the recommendations of the review committee of 2007 led 



NVAO to focus on structuring its internal quality assurance 
(IQA). 

Starting points of the NVAO IQA system 

NVAO has developed a system for internal quality assurance 

that: 

> applies the criteria of the EFQM-model for an integrated 
approach concerningthe organisation and its working 
processes; Z7> 

> facilitates frequent evaluation of the results and a 
structured plan for improvement (by following the 
Plan-Do-Check-Act cycle); 

> is transparent, simple and non-bureaucratic through the 
application of a limited number of instruments; 

> provides the basis for an external (international) evaluation 
by the adoption of international standards in the develop- 
ment of the internal quality assurance system; 

stimulates involvement of all staff members, and in addition; 
guarantees involvement of all stakeholders by means of an 
open dialogue with these stakeholders. 

Elaboration of the system 

NVAO distinguishes thirteen quality areas within the internal 
quality assurance system: (1) Strategy, Policy and Leadership, 
(2-3)(lnitial) Accreditation - including Research Master's 
Programmes, (4) International Affairs, (5) Communication, 
(6) Legal Affairs, (7) Support Services, (8) AdditionalTasks, 
(9) Internal Quality Assurance, (1 0) Human Resources, 
(11) General Services, (12) Financesand (13) ICT 

The system can be summarized as follows. For each quality 
area, strategie objectives and targets are formulated on a 
two-year basis. A renewed quality scheme 28> based on the 
strategie goals guides the process in every area. At least every 
two years, evaluation data are gathered systematically by each 
quality area working group. An integral evaluation scheme 29 ' 
has been developed to coordinate all NVAO's evaluation 
activities. Stakeholders can be questioned efficiently in this 
way The evaluations are prepared by the quality coördinator 
of the specific quality area. At the end of the two year 
period a report of progress is produced for every quality 
area (attachment 7 contains an example). 
Primary processes (Accreditation, Initial Accreditation and 
International Affairs) are subject to a more thorough 

27> EFQM (European Foundation for Quality Management) developed a 

non-prescriptive framework for organisational management systems and 
designed for helping organisations in their drive towards being more 
competitive. The Model is regularly reviewed and refined: the last update 
was published in 2010. 

28> attachment 6 

29> attachment 8 
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evaluation than other processes. The IQA group facilitates the 
process and steers the evaluations for the whole organisation. 
Every two years, after com pietion of the quality cycle, a quality 
report is drawn up, based on an evaluation per quality area. 
The next quality report will be available in 201 2. 
Accountability 

NVAO publishes its annual report every Spring. In the annual 
report NVAO presents its operational and financial results 
and activities of the previous year. 

Evaluation 

The integral evaluation scheme structures NVAO's IQA 
evaluation approach, and consists of written and verbal 
activities. Important NVAO evaluation structures / activities 
are: 

> The periodic meeting with the Advisory Council (three times 
a year). 

> The periodic meetings of the two- and four-party platforms 
in Flanders: the Flemish cabinet and higher education 
departement with NVAO (three times a year) and VLIR, 
VLHORA.the Flemish cabinet, the Flemish HE department 
and NVAO (three times a year). 

> The periodic meetings of the Dutch ministry and NVAO 
(approx. ten times a year) and the quality assurance 
agencies in the Netherlands (plenary twice a year, bilateral 
once a year); 

> Written evaluation of procedures initial accreditation; 

> Verbal evaluation of the procedure institutional audit and 
the accreditation procedures of the same institution. 

> Yearly survey of the external communication; 

> Yearly survey of international activities; 

> Yearly staff meeting to discuss the realisations of previous 
objectives and plans and formulate the new objectives 
within the PDCA-cycle of NVAO's internal quality assurance. 

Substantial evaluation activities in recent years were: 

> The evaluation of the accreditation system in the first phase 
in Netherlands and Flanders and the pilots to prepare the 
second phase; 

> Avisitof NVAOtoall Flemish higher education institutions 
in 201 1 to discuss all issues regarding (initial) accreditation 
processes asfaras NVAO is involved.The results were 
presented in the document Evaluatieve gesprekken in 
Vlaanderen; 30 * 

> An extensive survey of the staff's satisfaction in 201 1. 



Improvements in recent years 

A number of substantial improvements in NVAO's operation, 
achieved in recent years, are listed here. 

> (Initial) Accreditation 

- A new and evolved accreditation system in the 
Netherlands (201 1) and expected in Flanders (2013). 

- Certification of about 1 85 external secretaries (201 1 
and 2012). 

> Internationalisation 

- The certificate distinctive feature Internationalisation, 
which was developed and tested in a pilot (2010), leading 
to a new framework regarding the distinctive feature 
Internationalisation (201 1). 

- Six agreements with international partners regarding 
mutual recognition of accreditation decisions (2007). 

- The implementation of Crossroads (2009). 

> Information Communication Technology 

A new document management (DMS - RICOH, 201 1) and 
a new document administration system Easydoc (2008). 

> Communication 

- A new NVAO website (201 1), intranet (2009) and the 
quality magazine Q&A (201 0). 

> Human Resource Management 

- A Staff satisfaction survey and competence profiles for 
staff members (2011). 

> Strategy 

- A new Strategie Policy Statement 201 2-201 6 (201 2). 

> Organisation 

- The introduction of thefunction and rankof thematic 
coördinator, regular meetings with the director (201 0). 

Self-evaluation 

Since about three years NVAO employs a systematic 
evaluation system. All core assessment procedures are 
subject to periodic evaluation. The evaluation activities are 
on track and sometimes adjusted because of efficiency 
reasons. NVAO intended to develop a lean and, especially, 
practical IQA system. The current system is a result of this 
approach. In the opinion of NVAO an IQA system has only 
significant value if supported by a strong quality culture. 
Motivated, critical and proactive staff may even be more 
important to achieve NVAO's objectives. Therefore, NVAO 
invests in the quality of 'people and processes'. 

Judgement 

NVAO complies with the Standard. 



30> Evaluation meetings in Flanders: a thematical summary of facts and consider- 
ations of NVAO, 21 October 201 1, [Evaluatieve gesprekken in Vlaanderen: 
thematische samenvatting van de bevindingen en overwegingen van de NVAO]. 
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Point(s) of attention 

> Legal procedures are time and energy consuming, as was 
proved in the past years on several occasions. During an 
appeal procedure accreditation of a programme is 
suspended according to the law, which causes substantial 
uncertaintyforall involved and undermines the meaningof 
accreditation. 

> For NVAO (supporting) staff members the importance of 
their own contribution to IQA is not always clear. Better 
communication and more investment in commitment are 
necessary. 

4.9/ Miscellaneous 

ENQAcriterion 8 I 

The agency pays careful attention to its declared principles at 
all times, and ensures both that its requirements and 
processes are managed professionally and that its judgements 
and decisions are reached in consistent manner, even ifthe 
judgements are formed by different groups. 



NVAO is of the opinion that it has proven to be compliant with 
this criterion in the chapters three and four. 

ENQAcriterion 8 II 

Ifthe agency makes formal quality assurance decisions, or 
conclusions which have formal consequences, it should have 
an appeals procedure. The nature and form of the appeals 
procedure should be determined in the light of the constitution 
of the agency. 

NVAO is of the opinion that it has proven to be compliant with 
this criterion in the chapter 4.8.1. 

ENQAcriterion 8 III 

The agency is willing to contribute actively to the aimsof ENQA. 

NVAO is of the opinion that it has proven to be compliant with 
this criterion, especially in chapter 8.4. 
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In this chapter NVAO's position in the accreditation system 
in the Netherlands and Flanders is concluded by discussing 
the main strengths of NVAO's operation and the main points 
of attention.This summary is based on the three angles 
mentioned in the introduction: 

> NVAO as organisation within the new accreditation system; 

> NVAO's compliance with the ESG resp. Parts 2 and 3 and; 

> A first reflection on the new accreditation system in the 
Netherlands, which is presented separately in chapter 7. 

5.1 / NVAO as organisation within the 
new accreditation system 

Strengths 

Respected 

NVAO is a respected player with substantial impact on quality 
and quality assurance in higher education in the Netherlands, 
Flanders and in the European higher education area. As 
supervisory body in higher education in the Netherlands and 
Flanders NVAO holds an influential position. NVAO's initiating 
role in the evaluation and development of the accreditation 
system in the period 2008-2010 is just an example of its 
critical, self-reflecting,flexible, cooperative and transparent 
way of functioning, which have all contributed to the respect 
for NVAO in the eyes of all stakeholders. 
lts major contribution to the European Network of Quality 
Assurance (ENQA) and the European Consortium of 
Accreditation (ECA) are just two examples of NVAO's 
significant international contribution and NVAO is firmly 
positioned world-wide as a secretary of INQAAHE. 

Flexibleand pro-active 

Since 2005 NVAO has completed a large number of additional 
tasks as listed in chapter 8.5. All these tasks were assigned 
to NVAO on request of the Dutch and Flemish ministers. 
In completing these tasks NVAO showed not only its flexibility, 
but it also rapidly augmented its expertise in the field of 
quality assurance in Dutch and Flemish higher education. 
The pro-active character of NVAO came to the fore during the 
evaluation of the previous accreditation system and the 
preparation of the new system in the Netherlands and 
Flanders (see chapter 7). 

Thanks to NVAO's (inter)national position new developments 
in quality assurance are quickly recognized, evaluated and 
sometimes transferred to the accreditation systems or frame- 
works in the Netherlands and Flanders. The introduction of 
the institutional audit and the focus on learning outcomes are 
two examples of how NVAO is constantly learning from new 
developments. The clear difference between intended and 
achieved learning outcomes in the new accreditation frame- 



work is a good example of NVAO's pole-position in the new 
developments. 

Cooperation 

NVAO communicates and cooperates regularly in an open 
and transparent way with all stakeholders by organising or 
participating in conferences, pilots and workshops in Dutch 
and Flemish higher education. New developments and 
instruments are always implemented in cooperation with 
the stakeholders through pilots, as has been done in the 
development of the new accreditation system. 

Points of attention 

Administrative burden 

The administrative burden of the accreditation system is an 
important issue among stakeholders, especially the institu- 
tions. In the acceptance of the new system all parties agreed 
on a reduction of 25% in administrative burden in programme 
accreditation. Now it is upto all to monitor actively whether 
this aim will be achieved. NVAO has to play a leading role here. 
NVAO should also monitor whether the aim of a decrease in 
additional assessments and additional questions will be 
achieved in the coming years. 

Consistency 

Considering the large number of applications from the 
Netherlands and Flanders every year, consistency in 
assessment and its own decisions is a permanent issue for 
NVAO and the panels. In this respect NVAO will continue to 
support the quality assurance agencies, the (NVAO) panels 
and to monitor its internal processes. A new element in the 
accreditation system in the Netherlands and, possibly, in 
Flanders too, is the judgement of the programme on a 
four-point scale, which makes consistency even more 
important. The new evaluation matrix and the new internal 
organisational structure of NVAO are supposed to be of help 
in its strive towards consistency. 

Additional tasks and time limits 
Overthe years NVAO has completed a large number of 
additional tasks, which have sometimes caused workflow 
problems. As a result time limits of regular applications 
could not be respected more than once. In alignment with 
the Dutch and Flemish ministry NVAO must set priorities to 
better manage its primary tasks and additional assignments. 

Added value of binational. nature 
Because of its binational nature NVAO tries to contribute 
significantly to Dutch - Flemish cooperation in higher 
education. But an important objective from the start in 2003, 
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the mutual recognition of degrees, has not been achieved so 
far.This is disappointing, although mutual recognition is up 
to both governments and not to NVAO. NVAO has to look for 
other opportunities to strengthen the cooperation in higher 
education between the two countries in the near future. 

5.2 / NVAO's compliance with the ESG 
resp. Parts 2 and 3 

NVAO is convinced it complies with the European Standards 
and Guidelines, Parts two and three, from ENQA. NVAO leaves 
ittothe review panel tojudgeon NVAO's degree of compliance 
with each ESG and in the final conclusion. 
Regarding two standards of the ESG NVAO would like to add 
the following considerations. 

ESG 2.6: Follow-up procedures 

The accreditation system in the Netherlands and Flanders 
does not include aformal follow-up procedure. NVAO is 
confident it fully complies with what it believes is the core of 
the guideline in this respect - the transparent evaluation of 
the implementation of recommendations made by review 
panels, also in the case of positive accreditation decisions. In 
case of negative or conditional positive decisions of NVAO the 
follow-up procedure is clear, as is outlined in chapter 3.6. In 
case of an unconditional positive decision it is the responsi- 
bility of the institution and/orthe programmeto implement 
the recommendations of the panel. In the following assess- 
ment and accreditation procedure the panel will assess 
whetherthe recommendations are implemented adequately. 
Accreditation decisions are taken periodically. In this kind 
of system a formal approach regarding the implementation 
of recommendations of accredited programmes cannot 
operate well. For reasons of trust this should be leftto the 
accredited programme. 

The new accreditation system in the Netherlands offers the 
possibility of conditionally satisfactory in the procedures 
for institutional audit and initial accreditation. In addition, 
NVAO decides on the length of the recovery period, on the 
basis of a recovery plan that guarantees the follow-up of 
recommendations. 

ESG 2.8: System wide analyses 
Up to now NVAO completed several comparative analyses 
within a cluster or domain, as is described in chapter 3.8 
and therefore NVAO complies strictly speaking with this 
Standard. This does not preclude that NVAO has to work 
on implementing its new policy regarding system-wide and 
comparative analyses. 



A good step is the system-wide evaluation in the Netherlands 
and Flanders of the operation of the accreditation system in 
the second phase. 

Overall conclusion 

NVAO is of the opinion that it is compliant with the Standards 
and Guidelines of ENQA, while it is aware of points of 
improvement regarding its work processes. 
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This chapter gives an outline of the higher education systems 
in the Netherlands and Flanders. First a description wiLL be 
given of the types of institutions and programmes, and 
secondly some key figures on higher education will be quoted. 

6.1 / Higher education in the Netherlands 

Two types of institutions offer higher education programmes: 
universities of applied sciences (41 [hogescholen] and universi- 
ties (1 4), which both are publicly funded.They can offer 
programmes with an academie or a professional orientation. 
However, academie programmes are mostly offered by universities 
and professional programmes by universities of applied sciences. 

Besides these regular public institutions, there are recognised 
private institutions in higher education [Rechtspersonen voor 
hoger onderwijs]. Recognised private institutions do not 
receive public funding. However, after having completed 
the special procedure Legal body higher education, these 
institutions are allowed to apply for and to offer accredited 
bachelor's and master's programmes. Currently about 60 
recognised private institutions offer accredited higher 
education programmes in the Netherlands. 

Students who wish to enter higher education generally need 
a degree issued by one of the following types of secondary 
education: 

1. VWO (pre-academic education), the highest level of 
secondary education, required for being admitted to 
universities; 

2. HAVO (higher general secondary education) and 

3. MBO-4 (the highest level of middle professional education): 
both give access to enter higher professional education. 

In the Netherlands, higher education consists of three cycles: 
bachelor, masterand PhD level. Bachelor's and master's 
programmes can be offered by universities of applied sciences, 
universities and recognised institutions. Only universities are 
allowed by law to award PhD degrees. 
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Orientation / level 


Bachelor 


Master 


Research 
Master 


Doctorate 


Programmes with an 
academie orientation 


wo- 

bachelor 


wo- 
master 


wo- 
master 


PhD 


Programmes with a 
professional orientation 


hbo- 
bachelor 


hbo- 
master 







3 1 > Universities of professional education of significant size present 
themselves abroad as Universities of Applied Sciences. Dutch 
professional higher education counts 41 such institutions. Introd 



In the near future a new degree will be introduced: 
the Associate Degree as a short cycle part of the bachelor's 
programme with professional orientation (expected in 201 3). 
Currently Associate Degree programmes are offered as pilots. 

6.2 / Recent developments in higher 
education in the Netherlands 

In 201 1,the Dutch Government issued a new strategie 
statement for higher education 32> , which emphasizes the 
need for a more ambitious study climate in higher education, 
with a focus on excellence.The new strategy imposed a 
number of requirements on institutes in higher education. 
Universities need to reinforcethe relation between research 
and education and develop a sharply distinguished profile. 
Universities of applied sciences are required to focus on the 
development of research and entrepreneurial skills in the 
programmes, on upgrading the achieved learning outcomes 
of the professional bachelor's programmes and a master's 
degree for all educational staff in 2020 (already 80% in 201 6). 

A National Qualification Framework for the entire educational 
sector (NLQF) was finalized in the Netherlands in 201 1. 
This followed the development and acceptance of the 
European Quality Framework in 2008. As a result of the 
Bologna agreement, the National Qualification Framework 
for higher education (NQF HE) was approved in 2009. It is 
compatible with the Qualification framework of the European 
Higher Education Area (QF EHEA). NVAO monitors the NQF 
higher education framework in the Netherlands. 

6.3/ Higher education in Flanders 

In Flanders, higher education is provided by institutions 
registered in HOR. Only these institutions can offer bachelor's 
and master's programmes. There are two categories: 

> Statutory registered institutions. These were already 
recognised by the Flemish or Belgian government before the 
introduction of the bachelor's and master's degree system 
in 2003.AU these institutions receive public funding for 
education and research. There are four different types of 
statutory registered institutions: universities (6), university 
colleges (21), postgraduate institutions and protestant 
faculties. 

> Registered institutions. Since 2004 some new providers 
have successfully completed a procedure for registration 
and, consequently, were recognised as higher education 
institutions by the Flemish government. 

32> [Kwaliteit in verscheidenheid, Strategische agenda hoger onderwijs, 

onderzoek en wetenschap, Ministerie van OCW, Den Haag, augustus 2011]. 
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Universities have the authority to award academically- 
oriented bachelor's and master's degrees and doctoral 
degrees. University colleges have the authority to award 
professionally-oriented bachelor's degrees and in associa- 
tion 33> with a university academically-oriented bachelor's 
and master's degrees. The Flemish government intends to 
incorporate these programmes within universities. The 
Flemish system does not offer a professional master's degree. 

Access to higher education 

In Flanders, the following secondary school diplomas or 
certificates give access to higher education: 

> secondary school leaving diploma 

> certificate of short higher education courses with full 
curriculum or; 

> a diploma or certificate recognised as equivalent under a law, 
decree, European directive or other international agreement. 

Higher education institutions may adopt special admission 
requirements to admit persons who cannot meet the general 
admission requirement. These individual admission decisions 
must be based on:the individual education level of the 
student, humanitarian grounds; medical, psychological or 
social grounds, assessed by the higher education institution. 

Higher education in Flanders consists of three cycles: 
bachelor's, master's and PhD level. 
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A new degree has been introduced and will come into effect 
in the near future, which is linked to the short cycle 
programme HB05. Like the Associate degree programme in 
the Netherlands, HB05 is a short cycle programme on level 
5 of the European Qualification Framework. 



33> An association is a legal body in which the cooperation between a 

university and one or more university colleges is officially established. 



6.4/ Recent developments in higher 
education in Flanders 

Recent developments in Flemish higher education are partly 
directed bythe policy statement 2009-2014of the Minister 
of Education, Youth, Equal opportunities and Brussels 34 '. 
From this statement several pending issues and new 
initiatives in Flemish higher education can be derived. 

> A major topic in Flemish higher education for years is the 
'academisation' 35> of academie programmes in university 
colleges. In this regard it can be mentioned that the 
academie programmes in arts will be offered by specific 
Schools of Arts within the university colleges. The legislation 
allowing these changes should be ready in 201 2. 

> An ongoing debate in Flanders is the extension of academie 
master's programmes with a duration of one year to a length 
of two years. Recently (November 201 1) the universities and 
university colleges offered their proposals to the Flemish 
Minister. Afinal decision remains to be taken in April 201 2. 
The institutions have to finance the development of these 
programmes for the longer duration themselves, because 
the government will not offer extra funding. 

> A related topic is the discussion about the reduction of the pro- 
gramme Medical training to six years (now seven) in order to be 
equivalent to similar programmes in other European countries. 

> A recent and significant development in Europe is the growing 
importance of short cycle higher education. In Flanders the 
HB05 programmes can be provided in the future by university 
colleges or regionalcentres for adult education. These 
programmes are considered to lower thresholds for enrol- 
ment in higher education. NVAO supported the preparation of 
HB05 by organising a pilot in 201 to assess potential HB05 
programmes. As a result of the pilot a report of learning 
effects 36> was produced and discussed with all stakeholders. 

> To increase participation in higher education new 
arrangements for disadvantaged groups will be created. 

> A last objective is related to Internationalisation. In 201 6 
at least 1 5% of the graduates of bachelor's and master's 
programmes should have completed a learning experience 
abroad of at least three months. Either a practical training 
in the professional field or a couple of courses or a part of a 
programme at a foreign institution for higher education. 



34> Policy Statement 2009-2014 of the Flemish minister of Education, 
November 2009, [Beteidsnota 2009 - 2014, Onderwijs, Samen grenzen 
verleggen voor elk talent, Pascal Smet, Vlaams minister van Onderwijs, 
Jeugd, Gelijke Kansen en Brussel]. 

35> Academisation is the development of certain programmes from university 
colleges to academie programmes within an association, which is a legal 
body in which the cooperation between a university and one or more 
university colleges is officially established. 

36> Pilot report Learning effects HB05, NVAO, December 2010, 
[Leereffectenrapport proefprojecten HB05, NVAO]. 
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National Qualification Framework and Domain 
Specific Learning Outcomes 

The National Qualification framework of higher education 
(NQF HE) was approved by the Flemish government on 18 July 
2008. Related to the Flemish NQF HE are the domain specific 
learning outcomes, which are developed by all suppliers of the 
same kind of programme and which must be validated by 
NVAO for the levels 6 and 7 of NQF. Level 5 is validated by 
AKOV 37> . NVAO was an observer in the process of making the 
manual on how to write domain specific learning outcomes. 
As a result, NVAO produced a procedure concerning validating 
such learning outcomes of new and existing programmes on 
levels 6 and 7. Validated learning outcomes are automatically 
recognised by the Flemish NQF and replace the prepared 
domain specific frameworks by assessment committees. 

The National Qualification Framework (Het Vlaams Kwalificatie- 
raamwerk) for the entire education sector in Flanders was 
launched on 30 November 2009. An important step, because 
application of the NQF makes it possible to compare all indi- 
vidual, acquired qualifications and competences in educational 
programmesjobs and other societal activities.The NQF is an 
important instrument in lifelong learning. NVAO has been involved 
in thedevelopment of the NQF and was a memberof the Steering 
Group of the procedure that connected the NQF to the EQF 

6.5/ Overview of the Dutch and Flemish 
degree programme 

The Dutch and Flemish types of programmes described in the 
preceding paragraphs are shown in table 4. 



Table 4 / Matrix of the degree programmes in higher education of the 
Netherlands and Flanders 



Level - EQF 
Orientation 


Associate 

degree /HB05 (5) 3S 


Bachelor (6) 


Master (7) 


Academie 
orientation (the 
Netherlands 
and Flanders) 




180 ECTS 


60or120or180 
or 240 39> ECTS 


Professional 
orientation (the 
Netherlands) 


1 20 ECTS 


240 ECTS 


60 to 90 ECTS 


Professional 

orientation 

(Flanders) 


90or120ECTS 


180 ECTS 
Advanced: 60 
ECTS 





37> AKOV [Agentschap Kwaliteit Onderwijs en Vorming] is the new agency 
within the department of education to deal with (recognition of) 
qualifications and quality from EQF level 1 to 5. 



6.6/ Some key figures in higher educa- 
tion in the Netherlands and Flanders 

From 2005 to 201 enrolment in higher education in the 
Netherlands grew from about 560.000 to 635.000 students, 
in Flanders enrolment grew from about 1 60.000 to 1 95.000 
students in this period. 

In 2005 35% of the labour force between 27 and 34 years 
old in the Netherlands held a higher education degree, in 
Flanders this percentage was higher at 40%. Both countries 
performed better than average in Europe. Short programmes 
in higher education in both countries are not included in 
the percentages. These percentages are expected to grow 
further 40> . Table 5 shows several figures at a glance. 



Table 5 / Overview of (rounded) key figures of higher education in the 
Netherlands and Flanders (2009-2010) 



Variables / Countries /Total 


The Nether- 
lands 


Flanders 


Total 


Inhabitants 


16750.000 


6.250.000 


23.000.000 


Universities 


14 


6 


20 


Universities of applied 








sciences (NL) and University 








colleges (FL) 


41 


21 


62 


Recognised private institu- 








tions (NL) and Registered 








institutions (FL) 


60 


7 




Other statutory registered 








institutions (FL) 




5 




Programmes 


3500 


1400 


4900 


Number of students 


635.000 


195.000 


830.000 


Professional education 


400.000 


90.000 


490.000 


Academie education 


235.000 


105.000 


340.000 


Male vs. Female students 


48 vs. 52% 


46 vs. 54% 





Attachment 9 contains these figures more in detail. 
Sou rees 

Decreet van 4 April 2003 betreffende de herstructurering van 
het hoger onderwijs; Flemish education in figures, 2009-2010, 
Flemish authorities Education and Training policy area; 
www.hbo-raad.nl; www.vsnu.nl; www.vlir.be; www.vlhora.be; 
www.nvao.net; Jaarboek Onderwijs in cijfers CBS, 2009, 
2010en 2011 

38> Will be formalised in the Netherlands and Flanders in the next years 
39> Just some medical programmes have a study load of 180 or 240 EC. 

In the near future 90 ECTS will be possible in FL. 
40> OESO, Education at a glance, 2007. 
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The current accreditation system in the Nethertands and 
Flanders is described and evaluated below. 

UntiL the end of 2010 the accreditation system in the 
Netherlands was based on programme accreditation, peer 
review and the opportunity for programmes to apply for 
distinctive (quality) features. In Flanders this system is still 
current until the end of the academie year 201 2-201 3. 
From 1 January 201 1 a new accreditation system came into 
force in the Netherlands. It is expected that a roughly similar 
system will be implemented in Flanders in the academie year 
2013-2014. 

At the core of both the previous and the new system is the 
assessment and accreditation of programmes. Institutions 
prepare a self-evaluation report and quality assurance 
agencies assess the existing programmes of higher education 
institutions on the basis of the NVAO accreditation framework 
and procedures. Most agencies apply their own procedural 
handbookor protocol for the assessment procedure. In Flanders, 
the institution applies for accreditation of a programme with 
NVAO, submittingthe assessment report of the panel 
produced and published by the evaluating agency. In the 
Netherlands the assessment report is published after 
completion of the accreditation procedure. 
NVAO checks the quality of the assessment procedure and the 
quality of the programme according to the assessment report. 
If this report leaves questions unanswered, NVAO requests 
additional investigation. Finally NVAO takes an accreditation 
decision. 

The initial accreditation procedure is carried out entirely by 
NVAO, appointing the panels that assess a new programme on 
the basis of the framework for initial accreditation and submit 
their report to NVAO. 

Before developing a new system in the Netherlands and 
Flanders NVAO decided to thoroughly evaluate the previous 
accreditation system in both countries. After having discussed 
the experiences on accreditation with the external stake- 
holders in 2007, NVAO proposed a draft model for the new 
system, combining institutional audits and programme 
accreditation. The draft model was discussed during an 
international seminar in January 2008 attended by partner 
accreditation organisations in Europe that already gained 
experience with a mixed accreditation model of institutional 
audit and programme assessment. Subsequently NVAO 
discussed the draft of the new system with external stake- 
holders in Spring 2008 and started a pilot to test the draft in 
Autumn 2008. The pilot consisted of an institutional audit and 
a limited assessment of a programme (focusing on the quality 
of content and the achieved learning outcomes) of the same 



institution. Nine institutions participated in this pilot, sixfrom 
the Netherlands and three from Flanders. The pilot was an 
important learning experience for the institutions involved 
and for NVAO. The results were presented and discussed on 
special conferences for all stakeholders organised by NVAO 
in March-April 2009. Afterwards NVAO presented a definite 
proposal to the ministers of Education in the Netherlands 
and Flanders. 

Based on the experiences gained during the pilot and the 
support of stakeholders a new system of external quality 
assurance was introduced in Dutch higher education on 
1 January 201 1 (one year later than intended). In Flanders 
the new system will start in the academie year of study 
2013-2014. 

Besides the use of less but more open standards, another 
novelty of the new system isthe institutional audit. In the 
Netherlands an audit committee assesses whetherthe 
institution is 'in control' of the quality and the achieved 
learning outcomes of its programmes. In order to answerthis 
question the audit committee assesses the adequacy of the 
strategy, policy and educational processes in the institution. 
During the institutional audit at least two audit trails are 
executed. Institutions that pass an institutional audit 
successfully or conditionally with an improvement assign- 
ment are entitled to submit limited programme assessment 
(LPA). In the case of a conditional decision, the institution has 
to comply with a limited set of conditions within three years. If 
a negative decision has be taken, all institution's programmes 
that were assessed in the limited mode must undergo an 
additional, extended assessment within one year. Institutions 
that do not opt for an institutional audit can only submit 
application based on extensive programme assessment (EPA). 

Flanders still has to decide about some elements of this 
system, but it seems clear already that the institutional 
review and LPA will be compulsory, although in the first round 
without formal consequences for the result of the institutional 
review. This first round is meant as an important learning 
exercise and will be evaluated mid-term (201 7-1 8) and at 
the end (2021) in order to give time to work out the next 
accreditation system. 

The new accreditation system offers more opportunities for 
institutions and programmes to acquire recognition for higher 
quality or a specific profile. In the Netherlands, the final 
judgement is based on a four-point scale (Excellent, Good, 
Sufficiënt and Insufficiënt); this scale is applicable both on 
the level of a study programme and on the level of the 
separate standards of the framework. The new accreditation 
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system contains a recovery period and an elaborated 
procedure for initial accreditation in the Netherlands; if a 
new programme is approved, NVAO can restrict the duration 
of the initial accreditation and subject it to conditions that 
have to be met within a fixed period. 

The Flemish government still has to decide on the application 
of the four-point scale on the level of the study programme. 
An approved new programme is accredited for the length of 
the programme and two years (e.g. the duration of a profes- 
sional bachelor programme is three years, so the period of 
approval is five years). And as mentioned earlier, a recovery 
period is already part of the Flemish accreditation system. 

7.1 / Self-evaluation 

The old and new accreditation systems both in the 
Netherlands and Flanders can be summarized in the 
table below. 



Table 6 



Item 


Old system 


New system NL 


New system FL 


Accreditation unit 


Study programme 


Study programme 


Study programme 


Quality assurance 


Study programme 


Institution 


Institution 


# Standards 


6with21 underlying aspects 


LPA:3 (open) 
EPA:16 
Institution : 5 


Study programme : 3 
(4) 41> (open) 
Institution : 5 


Clustered assessment 


No clustering (NL prof) 
Clustering (NL ac. & FL) 


Clustered assessment 
of programmes 


Clustered assessment 
of programmes 


Assessment organisation 


Initial: NVAO 

Study programme : QAA 


Initial : NVAO Study programme : 
NVAO orders QAA 
Institution : NVAO 


Initial: NVAO 

Study programme:VLUHR 42> 
or EQAR/foreign QAA 
Institution : NVAO 


Panel 


Completely QAA-controlled 


NVAO-controlled 


QAA-controlled with NVAO advice 


Site visit 


2 days 


1 day+prep. meeting 


day+prep. meeting 


Report 


public 


public 


public 




Possibility (not FL) to disapprove 


Possibility to disapprove 


Possibility to disapprove 


Recovery period 


(not NL) Recovery period 


Recovery period 


Recovery period 


Doubt 


Further information Executive 
arrangement (NL) Hearing (FL) 


Further information Conditional 
accreditation 


Further information 


Follow-up 


Not official 


After recovery period After 
conditional decision 


After recovery period 


System-wide analysis 


Only as additional and 
voluntary task 


Togetherwith inspection 


Evaluation system-wide analyses 
in decree 



41 > Four standards for only registered institutions adding internal QA. 

42 > VLUHR [Vloamse Universitaire en Hogescholenraad] . VLUHR is a merger 

organisation of the former quality assurance organisations of VLIR and 
VLHORA, which both are EQAR registered. 
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The previous accreditation system is generally accepted 
in both countries.The quite open frameworks are widely 
appreciated. These frameworks provide assessment panels 
with the required room to apply the peer review to its tuil 
extent.The system is built on programme accreditation, 
which is to the satisfaction of most stakeholders, especially 
the government and students. 

On the other hand, the previous system shows some 
weaknesses as well. In the opinion of the institutions the 
administrative burden of programme accreditation is too 
high. For each programme common institutional aspects, 
like facilities and internal quality assurance, which are often 
managed at a higher level in the institution, had to be 
assessed separately. Such assessments were considered 
excessively formal and, as a consequence, educational staff 
feit less involved in the assessments. 

NVAO never intended the frameworks and procedures to 
oblige programmes to produce a lot of documentation for 
the assessments. To the contrary, NVAO encourages the 
programmes to deliver concise self-evaluation reports 
supported by a Limited number of appendices. Programme 
management often feels uncomfortable with a concise 
self-evaluation and compensate their uncertainty by 
producing heaps of documents for the panels. 

Another weakness is the position of quality assurance 
agencies in the Netherlands. These agencies are private 
organisations that are commissioned by the institutions. 
This position has raised questions aboutthe required 
independency of the agencies and their panels. 

A complicating feature was the lack of a recovery period in the 
Netherlands underthe previous system, which put the panels 
under extra pressure to formulate a positive assessment. 
The consequences of a negative assessment - cessation of 
public funding and no admission of new students - are very 
severe and can be fatal for a programme. As a consequence, 
some panels tended to write less convincing reports with a 
positive conclusion when assessing weak programmes. 

In some cases NVAO had to carry out additional assessments 
to verify the panel assessment and the quality of the 
programme. Sometimes the panel report was rejected. It is 
obvious that additional assessments (or additional questions) 
are considered as administrative burden by the institutions. 
But to guarantee quality assurance NVAO had to apply these 
steps occasionally. 



In the new accreditation frameworks, the institutional audit 
will focus on aspects that are defined at institutional level and 
includes organisational processes on programme level. 
Programme assessments will focus on content and results 
through the achieved learning outcomes.This combination 
should lead to a reduction in administrative burden. 
Institutional policy and processes within programmes are 
not repeatedly assessed in this model. The reintroduction of 
the cluster approach in the professional higher education 
sector in the Netherlands contributes to the consistency of 
the assessment of individual programmes. The introduction 
of a recovery period improves the objective judgement of the 
panels. The independent status and the quality of the panels 
has to improve as well, since the criteria are stricter, with a 
strenger focus on international and educational experience 
of the panelists. Moreover, in the near future the panels in the 
Netherlands will be commissioned by NVAO instead of by the 
institutions. The first indications of the effects of the new 
Dutch system are positive. NVAO already received a number 
of very well-written and well-argued reports and some with 
a negative judgement, which might be a first indication that 
panels pass stricter judgements, without the burden of a 
possible shutdown of the programme. 

As mentioned, the previous system is still running in Flanders. 
New elements in the Dutch system are already part of the 
system in Flanders, such as a recovery period and the cluster 
approach in both sectors of professional and academie 
education. In the past the quality assurance agencies in 
Flanders were part of the umbrella organisations of the 
universities (VLIR) and the universities of applied sciences 
(VLHORA). But in 201 1 these agencies merged into VLUHR 
(Vlaamse Universiteiten en Hogeschotenraad) partly in order 
to perform their external quality assurance activities sepa- 
rated from the umbrella organizations VLIR and VLHORA. 
This separation should guarantee that the external quality 
assurance system operates independently from the umbrella 
organisations VLIR and VLHORA. Nevertheless in the new 
Flemish accreditation system institutions will also be able to 
work with foreign QAAs as long as these are EQAR-registered 
or recognized by the NVAO. 

Procedures used by VLUHR are based upon the NVAO 
accreditation frameworkand procedures. The independence 
of the assessment committees is guaranteed by the 
Recognition committee fErkenn/ngscomm/ss/eJ of the Flemish 
government. As this will be changed in the new accreditation 
system VLUHR will only decide on the composition of the 
assessment panel after an NVAO advice. VLUHR plays an 
important role in the composition of panels. As this 
organisation is leading in the programme assessment 
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process, the added value of NVAO in the accreditation process 
is not immediately clear for all institutions and this obliges 
NVAO to communicate clearly about its role in Flemish higher 
education. 

NVAO organises periodic conferences and visits to improve 
the involvement and consent of the Flemish institutions. Both 
instruments were applied once again in autumn 201 1 as NVAO 
organised its Flemish conference in November in Brussels 
and visited all institutions to evaluate the experiences of 
the institutions with NVAO. Both activities produced useful 
information to improve the operation of NVAO in Flanders 
and are explicitly included in NVAO's new strategie policy. 

7.2/ Conclusion 

NVAO has positive expectations about the results of the 
new accreditation system in the Netherlands and Flanders. 
Lessons are learned from the first accreditation cycle in both 
countries and are translated into an improved system that 
responds well to the needs and the current achieved level 
of internal quality assurance in the Dutch and Flemish 
institutions. The systems offer an adequate challenge for 
the institutions to work further on internal quality assurance 
on institutional and programme level. 

The systems in the Netherlands and in Flanders will be quite 
equivalent, with some clear differences, but after all and in 
practice more similar than in the first cycle. As a result of 
the distrust that followed recent quality incidents in Dutch 
professional higher education, the initial openness of the 
new Dutch system was restricted by additional requirements. 
In Flanders the involvement of the higher education depart- 
ment in the Bologna transparency agenda resulted in an effort 
to restrictthe openness of the standards with underlying 
specific criteria and proofs. The talks to come to a consensus 
on the Flemish framework opened up the standards again. 
The first audit reports from the Netherlands demonstrate that 
these audits have a considerably wider scope than just the 
technicalities of the institutional internal quality assurance 
system. The audits start from the strategie policy of the 
institution, which is precisely the focus of the Flemish audits. 
Nexttothe recovery period it will be possibleto rejectan 
assessment report both in the Netherlands and Flanders. 
Remaining issues in both countries are the reduction of 
administrative burden in the system and the consistency 
and fairness of the panels. 

Even beforethe new Dutch system was introduced.the 
Netherlands and Flanders have learned from each otherfor 
the better. In the Netherlands the test of the macro-efficiency 



of a new programme has been placed before the assessment 
of initial accreditation, following the example of the order 
of these procedures in Flanders. The otherway around, 
Flanders has learned from the Netherlands to introducé open 
standards and to move the authority to impose a recovery 
period from the government to the NVAO. The differences that 
will remain in the new accreditation system are the result 
of the different degrees of trust in the quality of higher 
education and of cultural differences. Yet, the ultimate aims 
of the systems are similar, and the differences provide inter- 
esting insights and deserve being investigated in order to 
learn from each other. 
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8.1 / Quantitative results 

The quantitative results of the accreditation process in the 
Netherlands and Flanders until 1 January 201 2, are presented 
in table 7. 

Table 7 



8.3 / The treaty between the 
Netherlands and Flanders 

NVAO is established as a binational accreditation organisa- 
tion by treaty in 2003 by the Dutch and Flemish Education 
ministers. The Treaty assigns the tasks of NVAO, its form of 
administration and its supervision. 



Accreditation results 

Number of applications 
(Initial) accredited programmes 
Regular 

Research Masters 
Negative decision 
Regular programmes 
Research Masters 
Panel report disapproved 
Regular programmes 
Research Masters 
Application withdrawn 
Regular programmes 
Research Masters 

Total - Not accredited 



The Netherlands 



3982 



3385 
233 



21 

7 



17 



274 
45 



364 



Flanders 

864 
772 

38 



54 



92 



As mentioned, the firstphase of the accreditation system in 
Flanders runs till the end of the academie year of study 
2012-2013. 

Apart from the activities concerning accreditation, NVAO 
received more than 400 applications for additional tasks. 
These tasks are specified in attachment 2. 
Results regardingthe internationalisation task are presented 
in chapter 8.4. 

8.2 / The assignment of NVAO 

The primary assignment of NVAO is to accredit existing 
programmes in higher education (accreditation) and to assess 
new programmes (initial accreditation) in the Netherlands and 
Flanders. NVAO can be requested to carry out additional tasks 
bythe Ministers of higher education in both countries if these 
assignments support or supplement NVAO's primary assign- 
ment. 43 ' Secondly NVAO is assigned to play an important role 
in internationalisation (see chapter 8.4). 



At the time of writing (201 1) adjustment of the Treaty to recent 
developments is prepared. Since two years the ministers of 
Education from the Netherlands and Flanders discuss the 
possibility of incorporatingthe mutual recognition of 
diplomas in the Treaty. A long expected and natural step would 
be finalized in that case. 

Another reason to adjust the Treaty is the recent constitu- 
tional merge of the Caribbean islands Bonaire, Eustatius 
and Saba with the Netherlands. Consequently, NVAO will be 
authorized to act on these islands in the near future. 

8.4/ International assignment of NVAO 

From the outset in 2003, NVAO was assigned the task to play 
an important role in internationalisation. In order to fulfil 
this international role adequately, NVAO formulated five 
long-term objectives that outline its international policy. 
These objectives are: 

> Playing an active membership role in international networks 
of accreditation and quality assurance organisations 
(INQAAHE, ENQA, ECA). 

> Facilitatingthe profilingof the Dutch and Flemish 
accreditation and higher education systems so as to 
strengthen the international position of the Dutch and 
Flemish higher education institutions. 

> Cooperating with other accreditation organisations in order 
to achieve mutual recognition of accreditation decisions. 

> Contributing to the creation of a European Qualifications 
Area where the competent authorities automatically 
recognise degrees from accredited programmes and 
institutions. 

> Pro-actively following up on international developments 
in quality assurance and higher education. 

The department Internationalisation of NVAO works 
continuously on achieving these objectives. Operational 
targets and activities relating to each objective are formulated 
in an action plan. 



43> The Treaty, page 3. 
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Below the activities of the department are described shortly. 

1. Active membership of international networks 
in accreditation and quality assurance 

NVAO is an active member of three international networks 
of accreditation and quality assurance organisations: 

> International Network for Quality Assurance Agencies in 
Higher Education (INQAAHE) 

> European Association for Quality Assurance in Higher 
Education (ENQA) 

> European Consortium for Accreditation in higher education 
(ECA) 

International Network for Quality Assurance Agencies 
in Higher Education (INQAAHE) 

INQAAHE is a worldwide organisation for quality assurance 
agencies (www.INQAAHE.org). NVAO coordinates the INQAAHE 
Secretariat and the NVAO Vice-Chairman is as Secretary also 
Board member of INQAAHE. In the autumn 
of 201 1 NVAO has organised a INQAAHE seminar jointly with 
ENQA. Almost 1 20 participants from all continents were 
present at this three-day seminar in Brussels. 

European Association for Quality Assurance in higher 
education (ENQA) 

Members of ENQA are quality assurance agencies 
from countries that are party to the Bologna process 
(www.enqa.net). The NVAO Chairman is also a ENQA Board 
member. NVAO participates in the annual General Assembly 
and is as a rule present at ENQA Workshops. Several NVAO 
policy advisers have followed the ENQA reviewers training and 
presented at ENQA workshops or seminars. From 2007 to 
201 1 NVAO was represented in the steering group of the 
Internal Quality Assurance network of ENQA. The yearly 
seminar of this network was organised by NVAO in The Hague 
in 2009. 

European Consortium for Accreditation in higher 
education (ECA) 

The ECA network currently consists of sixteen accreditation 
organisations from ten European countries 
(www.ecaconsortium.net). NVAO coordinates the ECA 
Secretariat since the start in 2003. The NVAO Chairman is 
alsoVice-Chairof ECA. NVAO participates in all ECA events, 
includingthe annual plenary ECA Workshop, and plays an 
active role in the four working groups of ECA, chairing two of 
them.The winter seminars of ECA were organised by NVAO 
in the past years. 

The ECA projects TEAM II (2008-2010), JOQAR (since 2010) and 
E-Train (since October 201 0) are coordinated by NVAO. 



2. Facilitating the international profiling of the 
Dutch and Flemish accreditation and higher 
education systems 

NVAO representatives took part in numerous international 
conferences and workshops in which they provide active 
contributions.The Dutch-Flemish accreditation system and 
the binational cooperation clearly set an example for other 
countries, considering the large number of international 
delegations that visited NVAO in the recent years. In 201 and 
2011 NVAO welcomed delegations from 31 countries from all 
over the world (Armenia, Aruba, China, Curacao, Cyprus, 
Denmark, East-Timor, Ethiopia, Finland, France, Germany, 
Ghana, India, Indonesia, Japan, Kirgizstan, Lithuania, Norway, 
Peru, Poland, Russia, Saudi-Arabia, Slovakia, South Korea, 
Suriname, Sweden, Switzerland.Tadzhikistan, Taiwan, 
Trinidad and Tobago). 

NVAO participated in the international project on learning 
outcomes CoRe2 and has presented its views and approached 
at international seminars and in the ECA Working Group 4. 
In 2009 NVAO prepared aframework for adistinctive feature 
internationalisation, which was tested in 201 in a pilot in 
which 1 2 institutions and 21 programmes in the Netherlands 
and Flanders participated. 

3. Mutual recognition of accreditation decisions 

Mutual recognition of accreditation decisions is one of NVAO's 
main objectives. Mutual recognition of accreditation decisions 
facilitates the recognition of degrees and international 
mobility. 

NVAO signed six bilateral agreements on mutual recognition 
of accreditation decisions with agencies in other European 
countries and one multilateral agreement with five other 
European partners on the mutual recognition of accreditation 
of joint programmes (December 201 0). 

4. Automatic recognition of qualifications 

In the period 2005-2006, organisations responsible for 
recognition of foreign qualifications and the accreditation 
organisations from the Netherlands and Flanders, Norway, 
Austria, Poland and Switzerland signed the Joint Declaration 
concerning the automatic recognition of qualifications. 
The declaration is based on mutual recognition of 
accreditation decisions and on the implementation of 
compatible national qualification frameworks in order to 
achieve the automatic recognition of qualifications. 
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5. Pro-actively following up on international 
developments in quality assurance and higher 
education 

NVAO maintains excellent contacts with the appropriate 
government officials in the Dutch and Flemish ministries and 
with the appropriate members of the international network 
organisations. 

Other international activities thatthe Dutch government 
assigned to NVAO were the assessment of programmes 
offered on the former Netherlands Antilles and Aruba. 
NVAO also participates in international projects in order to 
follow up international developments in specific fields. In 
201 and 201 1 NVAO supervised university colleges and their 
umbrella organisations in Croatia to built an accreditation 
system. NVAO will supervise a similar project in Armenia in 
2012. 

Finally NVAO coordinated the external review of ENQA of 
the accreditation organisation HSV in Sweden in 201 2. 

8.5/ Additionaltasks 

Apart from its primary responsibilities (i.e. (initial) 
accreditation), NVAO has been charged with several additional 
tasks over the years. These tasks have to be approved by the 
Committee of Ministers orthe respective departments of 
Education and should be compatible with NVAO's mission. 
Forthese tasks, extra funding is made available. In NVAO's 
Strategie Policy Statement it is set down that new tasks 
should have an obvious connection with quality assessment 
of programmes. 

NVAO's most important additional tasks are the assessments 
of applications for: 

> Associate degree programmes (up till now 300 applications 
over the past years); 

> Open system (5 applications); 

> Room for Talent (11); 

> Recognition of private higher education institutions (1 5); 

> Broadened programmes (30); 

> Training Schools (83); 

> Educational Minors (1 0); 

> Academie Teacher training primary education (6); 

> Management of the official Register of Higher Education 
of Flanders (hogeronderwijsregister); 

> Academisation (140); 

> HB05(5); 

> Extension of academie master's programmes (29). 



Other additionaltasks of NVAO are the assessment of domain 
specific learning outcomes in Flanders and the monitoring 
of NLQF, both are described in chapter 6. In attachment 2 
the additional tasks of NVAO are described more in detail. 
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9.1 / Follow up of recommendations 
regarding NVAO 

In the assessment report of the external review in 2007 the 
committee suggested several recommendations regarding 
the operation of NVAO. In this chapterthe follow-up by NVAO 
on these recommendations will be discussed shortly. 

The review committee formuiated the following 
recommendations. 

Recommendation on ESG 2.8 

NVAO is advised to give more attention to the production of 
system-wide and comparative analyses - also with regard to 
the functioningofthe accreditation procedures. More attention 
for these analyses will be beneficial for the 'information 
function' of accreditation, as expressed by umbrella 
organizations of institutes and by student organizations. 

During the last few years NVAO completed a substantial 
number of comparative analyses. In several cases the 
analyses were ordered by the executive board of NVAO. 
A well-executed analysis can support and validate the 
decision-making of NVAO in case of single applications, 
and it provides also a solid basis to start a discussion with a 
cluster of programmes about quality assurance and quality 
enhancement. In other recent cases NVAO was asked by (one 
of) the stakeholders to present a comparative analysis of a 
specific programme cluster and a discipline or domain. 
The comparative analyses of the Teacher training (cluster) 
and the Arts programmes (discipline or domain) are good 
examples of such requests. In both cases the Dutch Minister 
issued the request. 

A short overview of the main comparative analyses in the last 
few years includes: 

> Academie programmes Linguistics in NL (2007); 

> Academie programmes Sociology in NL (2007); 

> Professional bachelor programmes Industrial Engineering 
in NL(2009); 

> Academie programmes Educational Sciences in NLand FL 
(2009); 

> Professional bachelor programmes Teacher training primary 
education in NL (2009); 

> Professional bachelor and master programmes Fine & 
PerformingArts (2010) in NLand 

> Academie research master programmes in NL (201 2). 

Several comparative analyses are still in progress.for 
instancethe professional bachelor programmes Human 
Resource Management, Communication and Social Work 



in the Netherlands. An international comparative analysis 
on the quality and level of three professionally oriented 
programmes in the Netherlands and Flanders together with 
two foreign countries or regions is planned. 

Another kind of comparative analysis concerns the 
professional master programmes in the Netherlands in 2007. 
At that time NVAO had assessed these programmes for three 
years. As professional master programmes are a new type 
of programme in Dutch higher education since 2004, an evalu- 
ation was useful to share first experiences and good practices. 

The evaluation of the former system and of the pilots in 
order todevelop the next system are good examples of 
system-wide analyses of the system. 
In attachment 1 the completed comparative and 
system-wide analyses are described in more detail. 

Recommendation on ESG 3.8 

To strengthen its accountability procedures regarding the 
relation with stakeholders and to focus in its internat quality 
assurance system and to include relevant stakeholders in 
the general board that are not represented yet. 

Since February 2010 a former president of the European 
Students Organisation is a member of the General Board of 
the NVAO. A new member of the professional field is present 
as well. Members of the General Board however do not 
represent a sector or field, but are nominated purely on 
individual merits and operate purely in their personal name. 

The Advisory Council of NVAO includes students since 2007. 
Representatives of the student unions ISO and LSVB (in the 
Netherlands) and VVS (in Flanders) have joined the Advisory 
Council for some years now. 

Recently, in 201 1, a resonance group was established to 
evaluate the introduction and the performance of the new 
system in the Netherlands. Representatives of institutions, 
umbrella organisations and students are the main members 
of this board. 

In Flanders a four-party platform has been introduced. 
Representatives of the Flemish cabinet.the higher education 
department of the ministry, VLIR and VLHORA (the umbrella 
organisations of the institutions) and NVAO meet on a regular 
basis to discuss matters of quality assurance and the 
preparation of the new accreditation system in Flanders. 
Representatives of VLHUR, beingthe new Flemish QAA 
merger of the former quality departments of VLIR and 
VLHORA, and VVS are invited when appropriate. 
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NVAO focused in structuring its Internal Quality Assurance 
system since the last review in 2007. In chapter 4.8 the main 
characteristics, strengths and weaknesses of the current 
operational system are evaluated. 

9.2 / Follow up of recommendations 
regardingthe accreditation system 

The review committee of 2007 recommended on several 
issues regardingthe higher education system and the 
accreditation system in the Netherlands and Flanders. 
As the accreditation system in the Netherlands and Flanders 
as such is not subject of this review the follow-up on the 
recommendations are only described here in relation to a 
first reflection on the new accreditation system (see also 
chapter 7) and more specifically, in attachment 1 0. 



AU in all.the conclusion of the self-evaluation of the new 
system in chapter 7 can be repeated. Over the years, and 
surely in the new accreditation system, the Netherlands and 
Flanders have come closertogether, if not in system and 
regulations, then in practice. Even before the new system was 
introduced, the two countries have learned from each other 
for the better. The order of testing the macro-efficiency of a 
new programme before initial accreditation is a good example. 
The difference that keep existing in the new accreditation 
system are results of the different degrees of trust and of 
cultural differences. Such differences provide interesting 
insights and deserve being investigated in order to learn 
from each other. The fact that the first round of institutional 
reviews in Flanders will not have legal consequences and 
will be evaluated can provide input for improvements to the 
Dutch system. 



The Committee recommended further harmonization of 
regulations and procedures between the two countries with 
regard to the categories in the table below. As the table makes 
clear on almost all recommendations progress is being made. 
The new accreditation system complies with a number of 
recommendations. 



Table 8 



2007 recommendation 


New system NL 


Legal protection of titles 


Recommendations of minister to be decided 




Will be checked in external QA 


Sanctions in the case of a negative 


Recovery period 


accreditation. The Flemish system (of a 




statutory repair period) should also be 




implemented in the Dutch system 




Position of macro-efficiency check 


Macro-efficiency check for programmes by 




official institutions 


Length of accreditation cycles 


6 years 


Clustered assessments on binational scale 


Clustered assessments on national scale 


Time limits & sanctions 


Time limits (6 months) with fines unless warni 


Relationship NVAO-QAAs 


NVAO instead of institution orders QAA 



New system FL 



Future concept of self-accrediting institutions Institutional audit possible first step 



Only with accreditation and strict use of Arts and 
Sciences only with academie programmes 
Recovery period 



Macro-efficiency check for programmes by 

official institutions 

8 years followed by 6 years 

Clustered assessments on national scale 

Flexible time limits (4 months) without fines 

NVAO communicates and supports QAA on 

quality (protocol) 

Institutional audit possible first step 
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Attachment 1 Comparative and system-wide analyses 



Analysis Academie programmes Linguistics (2007 f 

In 2007 the executive board of NVAO decided to start a pilot regarding the evaluation of the assessment and 
accreditation of the academie programmes Language Sciences in the Netherlands, in order to contribute to the 
enhancement objective of the accreditation system. The analysis focuses on strengths and weaknesses that 
programmes in this cluster have in common and of the approach and execution of the assessment by the panel. 
The report was discussed in the meeting of the NVAO board in September 2007 and was used internally as a try 
-out to discuss what NVAO's policy and practice regarding comparative and system-wide analyses should be. 

Analysis Academie programmes Sociology (November 2007 f 

The executive board decided to start another meta-analysis of academie programmes in 2007. The cluster 
Sociology in the Netherlands was analyzed, first because of the low quantitative study results of the programmes, 
but also because of a lack of ambition on the side of the institutions. It was feit that this lack of ambition reflected 
both student and teacher attitudes and was not influencing quality of graduates in a positive way. The report was 
published in November 2007 and the results were discussed with the deans of the faculties of all involved 
universities during a meeting in Utrecht on 27 November 2007. 

Conference professional master programmes and seminar (2007) 

On 21 June 2007 NVAO organised a conference Professional master programmes. At that time NVAO had 
assessed these programmes for three years. As Professional master programmes are a new type of programme 
in Dutch higher education since 2004 a evaluation was useful to share first experiences and good practice. During 
the conference four approved applications by NVAO from different disciplines were presented as examples of 
good practice, namely examples of programmes from the sectors Health, Arts, Education and Natural Sciences. 
Furthermore the importance of professional master programmes for Dutch higher education and the professional 
field was emphasized by the umbrella organisation of the universities of applies sciences [The HBO-raad]. The 
conference was concluded by a forum discussion between four experts, contributing from different perspectives, 
and the audience of about 80 people. NAVO presented all presentations and the results of the conference on its 
website. 

Analysis Professional bachelor programmes Industrial Engineering (2009) 3 
In 2009 NVAO started an internal, comparative analysis of the professional programme cluster Industrial 
Engineering. The main objective was to apply a check regarding the internal consistency in decision making as a 
large number of programmes were involved. A second objective was to compare the quality of the programmes 
on some specific aspects as coherence, study results and assessment. The report was discussed in the meeting 
of the executive board of NVAO on 1 September 2009. The result was not shared with the involved institutions as 
the objective was mainly NVAO internal oriented to support NVAO's decision making process. 

Analysis Academie programmes Educational Sciences (2009) 4 

In 2009 NVAO completed the comparison of the academie programmes Educational Sciences in the Netherlands 
and Flanders. This investigation was initiated, because of the concern about the achieved learning outcomes, the 
lack of complexity and study load of the curriculum in the programmes Pedagogical Sciences [Pedagogische 
Wetenschappen] in the Netherlands. It turned out that in general the Flemish programmes are more intense than 
the programmes in the Netherlands. Especially the achieved study load and the achieved learning outcomes are 
at a higher level. The results of this study were used by NVAO for its internal process to get a better picture of the 
quality of these programmes in the Netherlands and Flanders. 

Analysis professional bachelor programmes Teacher training (primary education, 2009) 5 

The programmes Teacher training for primary education have been subject of public debate in the Netherlands in 

2009. Social and political concern raised about the expertise of teachers in primary education, especially 



1 [Meta-analyse cluster Taalwetenschappen, September 2007\ 

2 [Meta-analyse cluster Sociologie, November 2007\ 

3 [Analyse Technische Bedrijfskunde, September 2009] 

4 [Breedteanalyse Agogische en Pedagogische Wetenschappen Vlaanderen en Pedagogische Wetenschappen en 
Onderwijskunde Nederland, December 2009] 

5 Systeembrede analyse hbo-bachelor Opleiding tot leraar basisonderwijs, 14 Oktober 2009 



regarding their elementary skills related to calculating and Dutch language. As a result, on request of the Minister, 
NVAO started a meta-analysis on the Teacher programmes in primary education after completion of the 
accreditation procedure in these clusters. NVAO presented the results to the Minister, the Parliament and the 
involved stakeholders in higher education in 2009 and delivered through this report to the public discussion about 
the quality of these programmes. 

Sector-wide analysis "Analysis of Fine & Performing Arts Assessment Reports" (2010) 6 
The request from the Ministry of Education, Culture and Science (OCW) to the Netherlands and Flanders 
Accreditation Organisation (NVAO) to describe the developments in Fine & Performing Arts programmes 
(professional bachelor and master) resulted in an extensive and in-depth analysis of 52 assessment reports in the 
disciplines of Fine Arts, Design, Dance, Theatre and Music. 

This sector-wide "Analysis of Fine & Performing Arts Assessment Reports" produced a positive picture of the 
quality of these arts programmes. This picture reflects the thoroughness of the programmes and is underscored 
by several strengths including a profession-oriented approach, individual artistic development, technical skills in 
the field, up-to-date input and a wide range of expertise on the part of the teachers, differentiation in the 
programmes offered and stringent selection for admission. In addition, several points for attention regarding the 
theory courses, the research component, the preparation for "entrepreneurship" and the formal and structural 
involvement of alumni and the professional field, indicate issues that could raise the quality of these arts 
programmes with additional attention and effort. These matters require a result-oriented approach. 
On 1 6 September 201 NVAO organised a seminar for representatives of the Arts programmes to share the main 
facts, findings and conclusions and to stimulate the follow up of the recommendations. 

Later on, the report provided a good reference to the committee Dijkgraaf, which formulated a new sector plan for 
the Arts sector (201 1), commissioned by the umbrella organization of the Dutch universities of applied sciences 
[HBO-Raad\. 

Review Research Master 2011 

In 2010 NVAO started the review project Research Master. The Research Master is a separate category of 
academie master's programmes focusing on the role of academie researcher. The length of these master's 
programmes is two years (120 EC). Part of the review is an International Thesis Assessment Study that was 
completed in Spring 201 1 and presented during a NVAO seminar on 24 March 201 1 . A panel of experts assessed 
74 master thesis, 37 of graduates of research master programmes and 37 thesis from graduates of regular 
master programmes in the same discipline. Within the same discipline all thesis of research master graduates 
were assessed with a higher note. The assessors were unaware of the origin of the thesis, belonging to research 
master or the regular group and unaware of the result of the thesis, achieved by the student. In the opinion of the 
assessment panel the research master theses are in general 'good or excellent' and can meet international top 
standards. 

In the investigation participated the programmes Cognitive and Clinical Neurosciences (Universiteit Maastricht); 
Economics & Business Administration (Universiteit van Tilburg); Communication Science (Universiteit van 
Amsterdam) en Media Studies (Universiteit van Amsterdam). 

The results were presented during the seminar Excellent profiling in The Hague on 24 March 201 1 attended by 
about 125 representatives from higher education institutions. Later a publication was completed (Peaks In Sight). 



[Analyse visitatierapporten Kunstvakopleidingen, mei 2010]. Available in Dutch on the NVAO website 



Attachment 2 Additional tasks NVAO 



Associate degrees 

A substantial additional task of NVAO since 2006 concerns the assessment of Dutch applications for Associate 
degree (Short cycle) programmes. More than 300 applications of these 'pilot' programmes were assessed by 
NVAO at the request of the State Secretary of Education in the Netherlands in eight rounds of execution in the 
period 2006 - 2012. These programmes can be provided by universities of applied sciences or recognised 
institutions which offer accredited bachelor's programmes in the same field of study. Associate degree 
programmes are short programmes in the professional bachelor's programme that is usually completed within two 
years. After completion of these programmes graduates can enter the labour market or continue their studies in 
the bachelor's programme of the university of applied sciences. It is expected that Associate degree programmes 
will be definitely legalized in the Dutch act in 2013. The graduates of the running Associate degree pilot 
programmes receive already the Associate degree, as has been regulated by law, and which situates at level 5 of 
the European Qualifications Framework (EQF). 

'Open system' 

A second extra task in the last years concerned the initial accreditation of programmes within the Dutch 
experiment 'Open System' [Open Bestel]. These programmes are offered by either a recognised (private) 
university of applied sciences or a recognised (private) institution for higher education, which under the terms of 
this experiment can receive public funding. The objective of the pilot is to investigate the (dis)advantages of a 
level playing field for funded and recognised (private) higher education institutions. The results of the experiment 
are not available yet. 

'Room for Talent' 

Another additional task concerns the assessment of the Dutch experiment 'Room for Talent' [Ruim baan voor 
talent]. Under the terms of this experiment, programmes are allowed to raise tuition fees and enrol students on 
the basis of selection if programmes offer 'recognised and evident added value'. Another part of this experiment 
was that it aimed to enhance the development of honours' programmes. To assess 'evident added value' in 
programmes with selection of students and higher tuition fees and for the assessment of honours' programmes, 
NVAO developed assessment frameworks and set up an evaluation committee. The evaluation report of the 
experiment was published in 2007 7 . Partly due to the results of the experiment a new law concerning Room for 
Talent will came into effect on 8 July 201 1 . The secretary of state asked NVAO to develop a framework for the 
assessment of the distinctive feature Small and intensive education. A concept was discussed with stakeholders 
in August 2011. 

Recognition of private higher education institutions 

The Dutch State Secretary of Education, Culture and Science stipulated the policy regulation concerning 
recognition of private higher education institutions and requested NVAO to play a role in the recognition 
procedure. NVAO is involved in this procedure since 2004 and assessed 15 applications. In September 2010 the 
procedure was renewed by the ministry. For this purpose, NVAO developed a new protocol 8 , including the criteria 
that programmes have to meet to be eligible for recognition. Recognition means that in the Netherlands a higher 
education institution - with the exception of the right to public funding - obtains the same rights as a publicly 
funded institution. If an institution is neither publicly funded, nor recognised and applies for accreditation of its 
programmes with NVAO, the institution will have to apply for recognition first. 

Protocol broadened programmes 

The Dutch minister for Education, Culture and Science determines whether an institution's intention to combine 
two or more programmes already incorporated in the Central Register for Programmes in Higher Education 
(CROHO) leads to a broadened new programme. In this case, NVAO carries out a marginal assessment and 



Evaluation report Sminia Comittee, [Evaluatie van de Commissie Sminia in het kader van Ruim Baan voor Talent, 25 mei 
2007\ 

8 Procedure Initial Accreditation regarding new Legal bodies in higher education [Protocol verzwaarde toets nieuwe opleiding, 
augustus 2010] 



provides advice to the minister. For this purpose, the 'Protocol Broadened Programmes' has been developed. 
Since 2004 NVAO assessed 30 applications for broadening programmes. 

Higher Education Register 

Another NVAO task concerns the online development and database administration of the Higher Education 
Register in Flanders (HOR, www.highereducation.be). This website lists all the recognised bachelor and master's 
programmes in Flanders. 

Assessment Teacher Training Schools 

The concept 'school based teacher education' has made a rapid progress in the Netherlands. 'Education together' 
is an important innovation with a complex implementation. 

A Teacher Training School ("Opleidingsschool") is a partnership between a school for primary, secondary or 
vocational education and one or more teacher training institutes. In 2009 the Minister of Education asked NVAO 
to assess the Teacher Training Schools. NVAO accepted this assignment and assessed 83 Teacher Training 
Schools. For this purpose NVAO developed an evaluation framework and composed an assessment panel that 
advised NVAO about the achieved learning outcomes of the graduates and about the partnership between the 
involved schools and the teacher training institutes. 

All Teacher Training Schools have been visited by two teams of experts: one team advised the NVAO about the 
achieved learning outcomes, another team advised the NVAO about the partnership between the Training School 
and the Teacher programme(s). 

Based on the assessment results, NVAO advised the Minister about the quality of the Teacher Training Schools. 
Three Teacher Training Schools were assessed as excellent, 24 as good, 30 as sufficiënt and 26 as insufficiënt. 
An additional result of the assessment was that success factors for Teacher Training Schools could be indicated, 
as the importance of real partnership, controlled growth of Teacher Training Schools, personnel policy, internal 
quality assurance and an inspiring learning environment. 

Assessment of Academie Teacher training primary education 

Academie Teacher training primary education is a special programme for students, attending an academie 
bachelor programme educational sciences, with interest in primary education. These students can follow at the 
same time the professional bachelor programme Teacher training primary education. This dual programme lasts 
four years (240 EC) and was introduced by the minister to enlarge the number of teachers in primary education 
and to increase the quality of graduates Teacher primary education. NVAO was asked by the minister to advise 
him on the quality of these dual programmes. In the years 201 and 201 1 NVAO assessed six programmes. 

Assessment of Educational Minors 

Educational minors offer the opportunity to students of academie, discipline oriented bachelor programmes to 
obtain for a teaching qualification. In 2009 the Minister asked NVAO to advise regarding the quality of plans for 
Educational minors. NVAO developed criteria and a limited framework and advised on 10 applications. 
Research in February 201 1 showed that the running Educational minors do meet the expectations of the 
graduates and the schools, where they start to work. 

Applications Extension of academie master's programmes 

Academie master programmes in the Netherland last mostly one of two years (60 or 120 ECTS). Programmes of 
60 ECTS can apply for extension to 120 ECTS, when they are of the opinion that the intended learning outcomes 
cannot be achieved by a curriculum of 60 ECTS. It is up to an expert panel and NVAO to formulate an advice and 
to the minister to decide. Up to now NVAO assessed 29 applications of this kind. 

Academisation 

In Flanders the former programmes of second cycle from the university colleges have to be transformed to 
academie bachelor's and master's programmes. These programmes are to develop to full academie programmes, 
but are professionally oriented as well. This reform has been running step by step since 2004. Special attention is 
spent to the development of academie research skills of students. In 2013 all involved programmes should meet 
the same requirements as the academie programmes of universities. NVAO incorporated the transition in the 
Accreditation framework and developed a manual Academisation and a manual Academisation Arts programmes. 



9 [Protocol ter uitvoering van de artikelen 7. 1 en 7.2 van de Beleidsregel doelmatigheid hoger onderwijs 2009 (Stcrt 2009, 1 15), 
inhoudende de uitgangspunten van de NVAO met betrekking tot de samenvoeging (verbreding) van opleidingen, de 
zogenoemde planningsneutrale conversie] 



In 2010 and 2011 NVAO decided on applications of 12 institutions forthe accreditation of 140 academie 
bachelor's and master's programmes in several disciplines, in which it was assessed to what extent the 
Academisation process has been implemented and whether it can be assumed that the process will be finished in 
the academie year 2012-2013. For each cluster of study programmes NVAO has been making an internal 
comparative analyses of the scores, textual appreciations and recommendations by the review panels in order to 
decide consistently on the possible conclusions (positive, negative, additional information, hearing or additional 
assessment). NVAO will decide on another substantial number of applications regarding academisation in 2012. 

HBÖ5 

Flanders will introducé a new level of (Short cycle) education on level 5 of the Flemish Qualification framework: 
HB05. It is a professional oriented programme, which does not offer a bachelor's or master's degree, but a 
graduate diploma. 

To support the implementation of HB05 programmes in Flanders NVAO developed a framework and organized a 
pilot to generate and disseminate learning experiences. Five institutions participated and their experiences were 
presented during a seminar 10 in December 2010. The results of the pilots are described in a publication 11 , which 
was handed to the Flemish Minister of Education, Youth, Equal chances and Brussels as well as to an audience 
of 270 people. 



10 Seminar HB05, Learning from pilots, 1 7 december 201 0, [Studiedag HB05 startklaar - Leren uit proefprojecten] 

11 Pilot report Learning effects, December 2010, [Leereffectenrapport - proefprojecten HB05\ 



Attachment 3 Workflow scheme - an example: Initial Accreditation NVAO 



1 


Registration of a new postal item (Archive) 


2 


Preparation of a new application file (Archive) 


3 


Application to be completed (adding information of the programme) (Archive) 


4 


To allocate the application in the organisation (Managing director) 


5 


Adding templates to application en sending confirmation of receipt and invoice (Secretariat policy advisors) 


6 


Checking the application for completeness (First responsible policy advisor) 


7 


Proposal panel composition (First responsible policy advisor) 


8 


Approval panel composition (First responsible Execute board member) 


9 


Adding panel composition to agenda meeting of the Executive board (Secretary meeting Executive board) 


10 


Report meeting Executive board (Secretary meeting Executive board) 


11A 


Informing institution aboutthe panel composition, informing panel, including the supply of the application file, planning 
site visit (Secretariat policy advisors) 


11B 


Analysing information file, organisation of the prepatory meeting panel and site visit (First responsible policy advisor) 


12 


Preparation of the intended decision and belonging documents for the meeting of the Executive board (First responsible 
policy advisor) 


13 


Approval intended decision (First responsible board member and lawyer) 


14 


Adding the intended decision to agenda of the meeting of the Executive board (Secretary of the meeting of the Executive 
board) 


15 


Report meeting of the Executive board (Secretary of the meeting of the Executive board) 


16 


Sending intened decision to the higher education institution (Secretariat policy advisors) 


17A 


Adding the application to the agenda of the General board (Secretary of the meeting of the General board) 


18A 


Report meeting of the General board (Secretary of the meeting of the General board) 


17B 


Preparation definite decision (Secretariat policy advisors) 


18B 


Control of all documents (First responsible policy advisor) 


19 


Sending definite decision (Secretariat policy advisors) 


20 


Publication definite decision and assessment report on NVAO website (department Communication) 


21 


Completion of the application file and all documents (First responsible policy advisor) 


22 


Control file by archive en provisional closure (Archive) 


23 


Definite closure or appeal (Archive) 



Attachment 4 Organisational chart NVAO 



Supporting services 

o Finance 

o Staff Management 

o ICT 

o Support 

□ Secretariat Executive Board 

□ Secretariat Director 

□ Secretariat Policy advisors 

□ Archives 

□ Reception 

o General Services 

□ Facilities 

□ Catering 



Board 
Executive Board 



Managing Director 



Policy 



Coördinators 
Policy advisors 
International Policy Advisors 



Advisory Board 



Staff Services 

o Legal affairs 
o Communication 



Legenda: 



-Formal-hierarchical line— 



- Advice line- 



Internal Consultation line ► 



Attachment 5 Strategie Policy Statement NVAO 2012 - 2016 



STRATEGY NVAO 2012 - 2016 

1. INTRODUCTION 

Now that NVAO has celebrated its first lustrum, the Committee of Ministers has evaluated and extended the 
NVAO treaty, and Flanders and the Netherlands are entering or have already entered the second phase of the 
accreditation system, NVAO wants to reassess its strategie goals for the next five years. This is not without 
reason: the social contexts for higher education in the Netherlands and Flanders have changed considerably the 
last few years and European developments are also moving forward. Higher education institutions have to adapt 
to these changing contexts, with students increasingly demanding higher standards and society critically 
appraising the achievements of institutions and students. This requires an active approach and responsive 
attitude of NVAO so that it can contribute to achieving optimal quality for higher education in Flanders and the 
Netherlands. For good education is the most important 'raw material' for both regions and NVAO may be 
expected to make a significant contribution to good education through its activities. The point of departure for this 
is that the responsibility for the quality of education lies first and foremost with the institutions. There can be no 
other way in a mature system of higher education. However, primary responsibility is not a carte blanche: 
education is too important and too expensive to be not monitored continuously. External quality control assures 
that the responsibility of the institutions is filled in appropriately. 

This Strategy Memorandum first continues with an overview of the most important external developments in 
higher education in the Netherlands, Flanders and Europe. Next, NVAO will present its mission, values and 
positioning. These will be translated into a number of strategie goals, after which the relationship with 
stakeholders, the internal organisation and the working methods of NVAO will be discussed. All this will result in 
the NVAO's 'strategie agenda' for 2012 - 2016. 

2. CONTEXTS 

2.1 Society 

Due to the importance of higher education for Flanders and the Netherlands, society wants to and should be able 
to trust the continuing good quality of higher education. Public interest in the achievements of higher education 
institutions has undeniably increased. Abuses in a small number of institutions, failing administrative procedures 
and complaints from students and lecturers have led to consternation and general indignation. As a consequence, 
doubts have arisen about the institutions' educational achievements and there is a call for more stringent 
supervision to make sure that each diploma has been obtained legitimately and that its holder has the expertise, 
skills and attitude expected of a recent graduate. Although this concern is understandable and these incidents 
indeed call for specific measures, the increased media attention for higher education appears to turn each 
incident into a structural shortcoming, as a result of which the institutions and their supervisors are subjected to 
an ever-increasing pressure. 

In Flanders, the public debate is dominated by recent developments, such as flexibilisation in higher education, 
academisation of colleges of higher education and their integration in universities as well as the consequences for 
the organisation and the quality of higher education as a whole. The institutions' difficult financial positions play a 
large part in this debate. 

2.2 Students 

By far the majority of students realise it is a privilege to take a degree programme at a university or college of 
higher education. This position comes with rights and obligations. In return for a substantial study commitment, 
students may expect to be offered high-quality education that is well organised. This is all the more cogent as the 
governments increasingly place the responsibility for and the costs of longer study duration and second-degree 
programmes with the students. For this requires the institutions to take even greater care of the content and 
process side of education. Moreover, institutions are expected to help students make appropriate study choices 
by providing good information and showing them at the earliest possible stage if they are suited for a certain 
study. This involves intake interviews, orientation tests, a (binding) study advice and at least good student career 
counselling. 



2.3 Ambitions 

Flanders and the Netherlands both realise that their economie growth and social-cultural power are very much 
driven by knowledge. The term 'knowledge economy' appears in every policy memorandum and strategy view. 
Moreover, both regions lack modesty - and quite rightly so - and show considerable ambition. The Netherlands 
has even expressed the ambition to rank with the top five knowledge-based economies in the world. Flanders, 
too, pursues an ambitious agenda in VIA. These aspirations require a large effort of the institutions, which are 
expected to compete with leading institutions in the United States, Great Britain and other western and emerging 
economies. For this reason, the institutions' achievements in education and research are closely followed. 

2.4 Diversity 

The belief that students cannot be regarded as one uniform and homogeneous group is taking firmer root all the 
time. The attention for differences in ability, social backgrounds and learning styles has increased strongly and 
leads to the conclusion that also in education more diversity is wanted and needed. The last few years the 
institutions have positioned themselves accordingly: this shows for instance in the creation - and flourishing - of 
the (liberal arts) colleges, the excellent master's track and honours programmes, preparatory programmes, track 
and study counselling, new broad programmes, but also associate degree and HB05 programmes, and a variety 
of deficiency courses. All this has led to a dramatic rise in the diversity of programmes offered. 
Moreover, this diversity is reinforced by the governments that also want the institutions to try and have different 
profiles from each other. Especially the Dutch government has turned this into an important issue acting on the 
so-called Veerman report. 

2.5 Rankings 

In the global 'market' for higher education and in the battle to attract top talent, the prestige of the institutions 
plays an ever-growing role. This prestige is largely determined by rankings. No matter how strongly one may 
object methodologically and sometimes also morally to these rankings that are solely based on research results, 
they do indeed play an important role and the institutions - and also the governments to a certain extent - have no 
choice but to take them into account. Universities, therefore, will have to carefully balance their attention between 
research and education, between investments in research and education. In doing so, it is not improbable that 
research will prevail. The colleges of higher education have managed to escape these rankings so far, but the first 
signs of wanting to make profiles and differences in quality transparent are already visible. 

2.6 Implementation of Bologna 

The policy implementations of the Bologna Declaration in 1999 have been formally completed in many European 
countries. An extraordinary amount of work has been done, but it is clear that formal completion does not equal 
material implementation in all cases. This is compounded by the fact that implementation did not everywhere take 
place simultaneously. Great national differences remain within the European Higher Education Area. The Bologna 
Process is continuing, however, working towards its final goal of realising one European area for higher education 
and research. After the introduction of the Bachelor Master structure, the implementation of the ECTS system and 
the Diploma Supplement as well as an independent external assessment system for the quality of education, 
attention has now shifted to bringing about national qualifications frameworks within a European context and the 
application of learning outcomes systems. In Europe, Flanders and the Netherlands lead the way in both areas. 
The last few years, there have been discussions about the most desirable system for external quality assessment 
in a considerable number of European countries. Although the direction of change is not unequivocal, a 
noticeable trend seems to emerge towards combinations of institution and programme assessments. The 
Netherlands and Flanders fit seamlessly into this trend and in a number of areas they have even set the trend for 
developments in other European countries. 

Special attention is needed for the evolutions of the European Standards and Guidelines (ESG) and the European 
Quality Agency Register (EQAR), two important outcomes of the Bologna Process. The ESGs put initial 
responsibility for standards on the institutions and they will be updated in the coming years. The other evolution is 
the careful expansion of the work area of EQAR registered agencies through national legislation. 

2.7 The EU 

In addition to 'Bologna', which is a non-legally binding agreement, the EU is increasingly taking initiatives in the 
area of higher education. In doing so, one cannot escape the impression that the EU wishes to take over the 
Bologna Process and is taking steps that will impact the national control of higher education. This applies in 
particular to the manner in which the EU promotes quality labels, aimed at the exercise of professions. This also 
applies to the increased attention and promotion of directives that set conditions for exercising regulated 
professions and have an effect on the content of education. And it also applies, possibly and to a lesser extent, to 



heavily sponsored EU initiatives to come to its own European classification (U-map) and ranking system (U- 
Multirank). This European ranking system continues to struggle with the difficulty of formulating sound indicators 
to measure the quality of education. 

Of course, the European activities are stimulated by the increased competition between the two large power 
blocks. A small number of Anglo-Saxon institutions of higher education still dominate the world's top research 
universities while Asian knowledge institutions are strongly on the rise. In relative terms, Europe has a much 
higher number of educational institutions of more than considerable quality, but if obviously feels challenged to 
take part in this competition and to take initiatives to also create more top institutions, within or across national 
borders. 

3. MISSION, VALUES AND POSITIONING 

Based on the Accreditation Treaty, the tasks and powers as laid down in laws, structural decrees and other 
general regulations as well as the contexts outlined in section 2, NVAO's mission is as follows: 

"NVAO is an independent and authoritative binational accreditation organisation set up by the Flemish and Dutch 
governments, whose primary goal it is to provide an expert and objective judgement of the quality of higher 
education in Flanders and the Netherlands. NVAO does this with a constructive, critical attitude, respecting the 
autonomy of institutions and their primary responsibility for the quality of their education, and with an open eye for 
the growing international context. NVAO is open, clear and transparent towards society and all concerned, 
especially the institutions of higher education and the students. " 

This mission translates into three main NVAO tasks: 

1 . assessing and assuring the quality of Dutch and Flemish higher education; 

2. promoting the quality of higher education by promoting a culture of quality, aimed at regular assessment and 
continuous quality increase; 

3. putting Dutch and Flemish sectors of higher education (institutions, programmes) on the map and 
strengthening their position by means of international cooperation. 

NVAO carries out these tasks based on the following composition of values: 
■ independence 
clarity 

respect for (the autonomy of the institutions and their primary responsibility for the quality of their education). 

Mission, tasks and values form the basis for the way in which NVAO will organise its activities in the coming 
years. In doing so, NVAO also chooses a clear positioning with regard to and for its stakeholders. First and 
foremost, this positioning will be marked by the necessary independence. For NVAO is neither the 'extended arm 
of the government' nor is it part of the sector of higher education. 

lts judgements have to be respected and trusted, which means that institutions have to feel treated fairly, that the 
government has to be able to trust the outcomes of accreditation surveys, that students and society have to be 
able to trust the value of the accreditation judgements and that other parties (for instance foreign institutions and 
students) know that accredited programmes deliver what they promise. The necessary trust in the quality of the 
accreditation process and its outcomes have to lead to authority: authority for NVAO, authority for the assessment 
panels and organisations, but also authority for the institutions and programmes that have been assessed and 
accredited. 

The positioning chosen by NVAO requires an active approach. Active when supplying information, active when 
pointing out shortcomings and areas for improvement, active also when taking position on issues regarding the 
quality of higher education. This active approach applies to both the national and international arenas. For NVAO 
explicitly chooses to put the position of Dutch and Flemish higher education on the international map. With this 
also comes the obligation for NVAO itself to constantly look for the best possible ways to fulfil its tasks. In the 
same way education is and should be constantly evolving, the accreditation system has to be adapted regularly to 
the requirements of the time, to circumstances and to good developments elsewhere. This requires permanent 
reflection on one's own functioning and the functioning of the system. 



4. STRATEGIC IMPLEMENTATIONS 



4.1 Assessment and assurance 

Clear and independent assessment 

It is essential and also an absolute minimum that the quality of education provided by the institutions and 
programmes is able to meet the (internationally accepted) standards of generic quality. By now institutions and 
programmes have had the opportunity to gain ample experience with the accreditation system. Their own 
professionalism and responsibility ought to teil them where to place the pass mark that constitutes a 'pass' or a 
'fail' and it is up to them to deliver good results. The most important evidence for this can be found in the achieved 
learning outcomes in final reports, portfolios and exams. By performing regular benchmarking through self 
evaluations and by calling in (inter)national experts during assessment procedures, it has to become clear that the 
achieved learning outcomes compare well internationally. It is with full conviction that NVAO chooses an 
assessment system that is supported by peers. In the new accreditation system even more demands are made 
on their qualities and they are either appointed (the Netherlands) or their appointment is recommended (Flanders) 
by NVAO. The quality of the peers and therefore the panels is decisive for the functioning of an assessment and 
accreditation system. The term 'peers' does not only refer to subject and disciplinary expertises, but also to 
expertises in the professional field, education and auditing as well as to international and student-related ones. 
Due to (the introduction of) a recovery period in the Dutch and Flemish systems, panels are now able to award a 
fail to standards when required with greater conviction: for in many cases programmes can be given more time to 
make the necessary improvements and thus raise their quality levels. This requires a completely independent 
approach of panel members, because the trust placed in the accreditation system stands or falls with the 
independence and clarity of the panel's judgements. 

The independence and clarity with which NVAO and the panels deliver their judgements are in principle based on 
trust. Trust in lecturers, students, board and management. For the point of departure can be no other than that 
each person concerned, in whatever role, makes the effort to deliver the best possible education. But even with 
respect and trust, the panel may come to a negative judgement. The motivation for this judgement - positive or 
negative - and the manner in which the recommendations for improvement are formulated, are crucial for an 
improvement incentive to the programmes. 

Explicit attention forpurpose of quality improvement in reports 

An independent assessment by authoritative peers does not only guarantee a good judgement, but it also 
provides opportunities to formulate recommendations and advice. Judging and advising are two different activities 
from essentially different perspectives and they ought to be separated carefully in the decision process and in 
reports. One of the lessons learned from the first phase of the accreditation system is that this distinction was not 
sufficiently made and was not deemed sufficiently important. As a result, the NVAO's panel judgements were 
sometimes regarded as less solid and recommendations were considered points of criticism. In the new systems 
in Flanders and the Netherlands and in the way panels and NVAO operate, the true distinction between judging 
and recommending has to be made apparent, thus strengthening the purpose of quality improvement of the 
system. 

Increasing the accessibility and information content of decisions and reports 

NVAO is aware of the fact that the sheer volume of the accreditation decisions and the assessment reports from 
the first phase of the system have rendered them insufficiently accessible to serve as a source of information for a 
wider public. This is regrettable, for the reports contain a wealth of information. In the new system, guidelines 
have been added concerning the public character of the reports (a summary of the judgement that is easily 
accessible), the obligatory enclosures with factual information (such as contact hours, staff-student ratio, returns) 
and a differentiated judgement on the level of standards and the total. This range of informative improvements 
should lead to more information for those concerned (in the first place the students) and those interested in higher 
education. This does require a meaningful and unequivocal defining of data, such as, for instance, success rates. 
NVAO will take initiatives to this end. 

Making differences in quality and profiling transparent 

When increasing the informative value of judgements, decisions and reports, it is also fitting to make differences 
in quality and profiling between institutions and programmes transparent. This has the strong backing of in 
particular the Dutch government, arising mostly from the need to make clear to students and the labour market 
what (institutions and) programmes stand for. A great help in all this is cluster assessment, which already exists in 
Flanders and is obligatory in the Netherlands as of 2012. Assessing on a four-point scale makes it possible to 



map out the differences in quality even more clearly. Not only a differentiated judgement but profile features, too, 
can help to have students make appropriate choices and to have employers choose from the correct pool of 
graduates. The profiles of the programmes can be expressed in the content objectives and the education process, 
and can be supported further by applying for distinctive (quality) features. Although programmes have not been 
very keen to do this so far, NVAO initiatives to arrivé at a distinction between the areas of 'internationalisation' and 
'entrepreneurship' have been greeted with enthusiasm by Dutch higher education at least. Besides, making the 
differences in profiling and quality transparent may encourage institutions and programmes to position themselves 
clearly. This, too, can contribute to the content development of and differentiation between fields of study. 

4.2 Promoting a culture of quality 

Making good use of institutional audits 

Important for the continuous improvement of quality is the attention for the creation of a culture of quality in 
educational institutions. For accreditation provides a random picture and it is not good if the attention of the 
programmes is only aimed at achieving the threshold for accreditation every number of years. A culture of quality, 
by contrast, is an organisational culture that is aimed at quality and its continuous improvement. For this reason, 
in the new accreditation system much attention is given to assessing the quality assurance system at institutional 
level. The question then is whether an institution is in control of the quality of its education, in the sense that the 
board of an institution has a clear picture of its education results, receives signals of inadequate functioning, is 
able and willing to take measures and pays attention to improvements. 

Vital for the creation and existence of a culture of quality is having an attitude that is critical and development 
driven with regard to one's own achievements: this goes for lecturers among themselves, this goes for students, 
management, the professional field and alumni. Collectively they all ought to guarantee the existence and 
improvement of the quality of a programme. Stimulating this critically constructive attitude is a prerequisite for the 
creation of a culture of quality. In this area, too, NVAO will act encouragingly, especially by spreading good 
practices. 

An active approach in promoting the quality of education 

NVAO wants to contribute to intensifying discussions about the quality of education. There are various tools for 
this. Dutch and Flemish legislators agree that NVAO should devote more time and attention to making analyses 
that cross programme boundaries. These can be analyses of clusters of programmes in a particular domain, but 
also thematic reports dealing with issues that politics and society find relevant. Of course, the aim of these 
analyses is to come to conclusions and to subsequently base recommendations for improvement on these. 
Moreover, NVAO will gain an insight into good practice, when the judgement of 'excellent' is awarded, for 
example, to standards and programmes. The same goes for distinctive (quality) features. NVAO will focus on 
these through conferences, debates, seminars and publications. lts role here is to facilitate and inspire the sector 
of higher education, offering institutions and programmes the opportunity to assume responsibility to come to 
quality improvement. 

4.3 International perspective 

As a bi-national organisation, NVAO has paid much attention to its international position from when it was 
founded. Both governments have encouraged this international profiling and it is also in keeping with NVAO's bi- 
national character. ECA, the European Consortium for Accreditation, was founded as early as 2003; NVAO is in 
charge of the secretariat of INQAAHE; through its board members, NVAO has a presence in the boards of ENQA 
and EQAR; collaboration agreements have been signed and projects have been carried out with numerous other 
countries. 

Obviously, NVAO is not internationally active for its own sake. Of course, its international activities have to serve 
goals that will be of use to Flemish and Dutch institutions of higher education. However, it is not always easy to 
make that direct use immediately obvious. But in the same way the Netherlands and Flanders wish to profile 
themselves as prominent knowledge economies, Flemish and Dutch institutions of higher education choose to 
have prominent international profiles, NVAO, too, will have to profile itself internationally, in the vanguard of 
quality assurance organisations. 

Good positioning of Flemish and Dutch higher education abroad 

On balance, Flemish and Dutch higher education is of good quality. This is also recognised abroad. That quality 
has to be underlined by the quality of the assessment systems used. This makes it necessary for NVAO to 
expound the quality of the accreditation system, especially by drawing attention to the unambiguous manner in 



which it judges the level aspired to by the institutions and that achieved by their students. The Netherlands and 
Flanders are ahead when it comes to assessing achieved learning outcomes. This practice is a prerequisite tor 
thorough international comparisons, but it will be a long time yet before other countries and quality assurance 
organisations will follow their lead. The binary character of Dutch and Flemish higher education fairly often 
requires an explanation in countries with a unitary system. Especially the Dutch 'professional master' is not known 
as a separate degree in many countries. Therefore, NVAO will take initiatives to underpin and promote 
international acceptance of this degree. 

Intensifying cooperation with reliable partners 

It is beyond doubt that institutions benefit from a system of accreditation that is considered solid and reliable by 
foreign institutions and quality assurance organisations. It increases trust in its quality. However, the broadness 
and strictness of the Flemish and Dutch system of accreditation, which is remarkable in an international 
perspective, occasionally lead to self-critical comments about our higher education, while foreign programmes 
and systems are a lot less critical with regard to certain aspects, such as for instance 'achieved level', which they 
may not even assess! So now and then it appears that a difference is arising between domestic criticism and 
foreign appreciation. This does not mean to say that the Standard of quality should be lowered, but it does mean 
that NVAO and the institutions have to communicate well and choose their partnerships carefully. 

Developing a set of instruments for promoting internationalisation 

Choosing carefully plays an important role, for instance, when it comes to the applaudable foreign mobility of 
students. Whereas during the first years of mobility promotion, academie tourism was by no means accidental 
when choosing foreign destinations, now quality requirements are set by institutions, programmes and students. 
This is why NVAO chooses to work with preferred partners (especially in Central and North-West Europe), but 
also looks for strong partners elsewhere in Europe by means of distinctive features, in particular in the field of 
internationalisation. In this way, NVAO also hopes to make an indirect contribution to one of the objectives on the 
Leuven agenda, namely promoting student and teacher mobility. 

Active contribution to comparability between programmes 

Student mobility can also be promoted as more and better data are known about the content of the programmes 
that are offered in the thousands of European institutions of higher education. Qrossroads continues to be an 
excellent instrument for this. But the possibilities to compare between programmes require more effort, which is 
why active involvement in determining and interpreting national qualifications frameworks is of great importance. 
In this area, too, the Netherlands and Flanders are leading the way in Europe. The same goes for the use of 
learning outcomes, for which Flanders has developed detailed procedures, which have been tested in pilots the 
last few years. In the Netherlands, too, learning outcomes will be translated into more specifically domain related 
ones. It may be a consideration to give learning outcomes a more prominent place in assessment and 
accreditation reports. 

Increasing activities outside Europe 

Of course, international cooperation cannot be limited to Europe. More and more institutions and programmes are 
active in Asia and the US, and this also requires intercontinental relations and agreements from NVAO. For the 
Netherlands and Flanders have major ambitions in higher education. External quality control ought to be in line 
with these ambitions and to promote and assure them. The NVAO's international activities and collaborations 
need to also serve this purpose. 

Relations have been established and sometimes formalised with organisations in Japan and China, while 
cooperation with American accreditation organisations takes place on an ad-hoc basis. Especially the elimination 
of degree mills requires close cooperation and a pattern of close relations. 

Enhancing the added value of Flemish-Dutch cooperation 

NVAO wishes to clearly emphasise and enhance the added value of the Flemish-Dutch cooperation. This added 
value is primarily evident in the detached manner with which NVAO can make its judgements due to its bi-national 
nature. This manifests itself in the way NVAO operates, with both Flemish and Dutch eyes looking at how 
programmes are judged. The same bi-nationality is reflected in the composition of the panels, the more so as 
soon as the Netherlands, too, will have introduced cluster assessments. But more is possible, not only in the 
cooperation across boundaries between institutions and programmes, tuning of domain-related learning 
outcomes, analysing education results in domains and disciplines, but also by simply exchanging information and 
good practices. In all these areas NVAO wants to organise more activities and thus strengthen Flemish-Dutch 
cooperation. However, it is with great disappointment that NVAO concludes that the existence of close Flemish 



and Dutch cooperation in the accreditation system has not yet led to unequivocal steps towards mutual Dutch- 
Flemish recognition of diplomas. With the accreditation system in mind this is hard to understand and not easy to 
explain either. For in both regions assessment of the bachelor's and master's levels takes place based on the 
Dublin descriptors and the generic and domain-specific learning outcomes that are based on these. The NVAO 
pronouncements ensure that bachelor's and master's levels are achieved. This should be the basis of mutual 
recognition rather than comparing the content and/or length of programmes. In the light of the intended creation of 
a European Higher Education Area, the Netherlands and Flanders should at least see to it that this issue is sorted 
binationally. 

5. RELATIONS WITH STAKEHOLDERS 

NVAO works in a comprehensive field. Not only is it confronted with two systems of higher education, but also 
with different political cultures and structures that give rise to different courses of action and positions. This means 
that the stakeholders in both systems have to be dealt with in a careful but sometimes different manner. Good 
communication is essential for this. 

5.1 Institutions 

Of course, the institutions of higher education are very important stakeholders. In its approach of the institutions, 
NVAO chooses for critical fairness: a critical, objective and respectful approach. This attitude is a result of the way 
NVAO sees its role, which has as its first main task the assessment and assurance of the level of Dutch and 
Flemish education. The institutions need to be well aware of this role: NVAO is neither there to 'settle scores' nor 
to cover up, but to assess if at least a generic level of quality is provided - and preferably more than that. 
The recommendations of the panels and the broader, more reflexive analyses by NVAO offer good directions to 
come to quality improvement at the level of an individual programme, institution, domain or theme. 

5.2 Politics 

The position of politics, as a derivative of society, as a stakeholder has changed considerably the last few years. 
On the one hand, this has to do with the aspirations of the governments to strengthen their knowledge 
economies, in which higher education plays a crucial role. On the other hand, politics has also become the self- 
appointed spokesperson of generally feit concern about noted shortcomings in quality. This leads to calls for clear 
pronouncements on quality, for identifying problems and closing leaks in supervision. Therefore it is necessary 
that NVAO, and consequently also the institutions, provide unequivocal information about those critical success 
factors that are considered decisive for the current quality and for the improvement capacity of institutions and 
programmes. The situation in the Netherlands, for that matter, is complex because both NVAO and the 
Inspectorate of Education have a role in the assessment and supervision of the quality of education. Although 
NVAO in principle is only concerned with assessing the content and results of education, and the Inspectorate's 
tasks have the application of rules and legislation as their departure point, an overlap between the activities of 
both organisations is almost inevitable. For this reason, the Inspectorate and NVAO have signed a covenant in 
which agreements have been made about the way in which the two organisations cooperate in order to come to 
an assessment system that functions to the best of its ability and also keeps the costs down for the institutions. 

5.3 Society 

The last few years NVAO has been relatively reticent in contributing facts or opinions to social debates. By 
consciously choosing to call itself an 'authoritative' organisation in its mission and by putting more emphasis on 
promoting quality improvement, NVAO will also adopt a more public stance. This political stance is especially 
aimed at quality improvement in higher education and strengthening a culture of quality. 
This comes with one caveat: more supervision and more rules do not automatically lead to better quality. NVAO 
also considers it its duty to point to the consequences of changes to the system and a new, greater need for 
information. The issue of 'reducing the burden of accreditation' is firmly on the agenda and the issue of a possible 
reduction in bureaucracy remains topical under all circumstances. 

5.4 Students 

Much of the NVAO's activities has to benefit students. It would be good if, before starting their studies, more 
students would act like well-informed consumers, making rational considerations regarding the future institution 
and programme of their choice. NVAO is going to actively contribute to this, not only by actively engaging 
students in all types of assessment panels, but also by taking care of good accessibility of information on the 
quality of the programmes. The latter increasingly plays a role in view of the growing call for realising a distinction 



between institutions and programmes. By also involving the institutions' students as well as possible in assessing 
the quality of education, they will evolve from education consumers into education participants. 

5.5 Evaluation organisations 

NVAO duly takes into account the functioning and quality of evaluation organisations. In Flanders the position of 
these organisations is defined by law, in the Netherlands they will have to qualify to fulfil an active role in the 
assessment and accreditation system. The quality of the system is strongly dependent on the work of the panels 
and evaluation organisations. Open communication, good feedback and high-quality participation from all those 
concerned will therefore have to give the system a solid foundation. 

5.6 Professional field and labour market 

The labour market, too, is increasingly interested in data on quality and profiling. This interest can be seen at the 
'back' of the system, from graduates, but certainly also at the 'front' when setting the intended learning outcomes 
or qualifications that students have to meet. By publishing clear reports and organising thematic meetings, NVAO 
can contribute to complying with requests for data on quality and profiling. 

5.7 International colleague organisations 

NVAO also considers its foreign colleague organisations as stakeholders. They should not only understand how 
the Flemish-Dutch accreditation system functions, but they should also be able to judge if the outcomes of the 
assessments are sufficiently reliable. Moreover, NVAO has to communicate and explain changes and 
developments in the Dutch-Flemish system. In international trust, two 'external judgements' play an important 
role: the ENQA assessment in 2012 to determine if NVAO meets the European Standards and Guidelines 
resulting in full membership of ENQA, and the subsequent assessment to determine if NVAO continues to be 
registered with EQAR. 

At the moment there are various forms of structural meetings with stakeholders: 

■ the binational advisory council: with representatives from institutions, students and the professional field; 

■ meetings with the Committee of Ministers, which take place twice a year; 

■ the Flemish sounding board meeting with umbrella organisations, evaluation organisations and the 

government; 

■ bilateral contacts with (umbrella) organisations, ministries, student organisations, employer organisations. 

In order to inform its stakeholders and other interested parties, NVAO yearly publishes an annual report as well as 
an account of income and expenditure, complete with a positive auditor's report. There are also other 
publications, a Newsletter, and NVAO has an up-to-date website with a search engine for finding topical 
information on programmes and institutions. 

6. INTERNAL ORGANISATION 

It is with a certain satisfaction that NVAO looks back on its achievements of the past 6 to 8 years: as agreed all 
programmes have now been assessed once (with the exception of a number of Flemish clusters that are still to be 
accredited in 2012). This means that some 4,000 existing programmes and over 500 applications for new 
programmes have been assessed. 

These activities have been complemented with all kinds of extra tasks, such as assessing Associate Degree 
programmes in the Netherlands and a HB05 pilot in Flanders, from programmes like 'training in the school' and 
educational minors in the Netherlands to domain specific learning results in Flanders, as well as executing all 
kinds of international projects. This has put considerable pressure on the organisation, both on its staff and 
finance. Yet the NVAO's regular budget has hardly risen the last few years, which is thanks to the great efficiency 
and effectiveness of and within the organisation. With the second phase of the accreditation system, the 
organisation is entering a new phase, but the internal professionalisation and the organisation of its business 
processes seem to be well prepared for this. 

It is becoming clear though that there will be a greater need for the analytical and reflexive abilities of the NVAO 
as well as its internal quality assurance system. The former is necessary to come to good analyses and 
recommendations that rise above programme level, the latter especially to implement the desire for differentiation 
as expressed in judgements in a well-motivated and consistent manner: after all, the institutions and programmes 
stand to benefit greatly from this. 



This means that also within NVAO much attention needs to be paid to the continuous development of its own 
culture of quality, with as its most important elements: 

■ permanent sharpness and a critical attitude 

■ collegial intervision and procedural purity 

■ consistency in judgements 

■ an active and reflexive attitude 

■ quick and institution-oriented processing of applications 

■ continuous professionalisation of people and work methods 

■ clear and reliable internal and external communication 

■ stimulating and directional management 

■ a slim, but efficiënt, internal quality assurance on all processes 

In this, the NVAO's General Board will act using its common sense and helicopter view as non-executive fellow 
director, sounding board and facilitator of general procedures. In order to support the introduction of the new 
system, NVAO has set up a sounding board group for the Netherlands, which will become operational as soon as 
the first reports on the institutional audits and the new programme assessments have been completed. 

In addition, NVAO is considering the creation of an international advisory council, which can apply its foreign 
expertise especially to detect possible problematic issues in the new systems in Flanders and the Netherlands 
and also to establish a link with developments abroad. This could be seen as an extra legitimisation as well as the 
start of thinking about the third phase of the system. 

For NVAO it has become clear these past few months that the enthusiasm with which new tasks and possibilities 
have been received, cannot be allowed to continue without restraint. Increasingly the question will be asked how 
new demands and developments will be able to contribute to the quality increase and transparency of the Dutch 
and Flemish higher education as well as our own operations, both internally and externally. This requires more 
focused choices, which will have to be in line with the strategie course as described in this memorandum. 

STRATEGIC AGENDA 2012 - 2016 

From the above mission, positioning and strategie elaborations, NVAO derives the following goals it wants to 
(continue to) realise in the coming years. 

Assessment and assurance 

■ clear and independent assessment of programme quality and of quality assurance at institutional level 

■ explicit attention for purpose of quality improvement of assessments reports 

■ increasing information accessibility of decisions and reports, including meaningful and unequivocal defining of 
core data 

■ making transparent differences in quality and profiling between programmes and institutions 

■ implementation and further development of new accreditation systems in the Netherlands and Flanders 

Promoting culture of quality 

■ good use of institutional audits 

■ active approach in promoting quality of education by means of: 

o encouraging institutions and programmes to act on recommendations in assessment reports 
o broader analyses of and reflections on system and domain level and theme analyses 
o meetings and seminars, aimed at opportunities for quality improvement in higher education and 
possibilities to realise these 

International perspective 

■ good positioning of the quality of Flemish and Dutch higher education abroad 

■ intensive cooperation with reliable and internationally leading partners 

■ development of instruments for international cooperation of institutions and programmes 

■ active contribution to increase comparability of programmes 

■ increase activities outside Europe 

■ (continue) to look for the best possible forms of external quality assessment 

■ increasing and clarifying the added value of Flemish-Dutch cooperation 

■ contributing to the mutual and international recognition of diplomas 



Clear positioning and relations with stakeholders 

■ positioning of NVAO based on mission and core values 

■ making explicit the balance between independent, clear and - if necessary - critical judgements on the one 
hand and promoting and facilitating the content of continuous quality improvement on the other hand 

■ external communication and a more public stance to realise the desired positioning and stakeholder relations, 
including the use of social media 

■ executing the covenant with the Inspectorate of Education 

■ continue with transparent reporting, in terms of both finance and content 

■ good relations with the Flemish and Dutch evaluation organisations 

Internal organisation NVAO 

■ adapting the office organisation to new/changing activities 

■ further development of a top-notch NVAO quality culture, including renewed international certification 

■ continuous professionalisation of all staff members 

■ frequent testing of internal and external work on strategie objectives through slim but efficiënt internal quality 
assurance 

■ making use of the strategie objectives as criteria for taking on additional tasks and (international) projects 

■ linking internal and external communication with this strategie policy plan 

FINALLY 

For NVAO, the 2012 - 2016 strategy means a different positioning. Whereas in the past few years the emphasis 
has been on executing primary tasks and assessing whether or not standards of basis quality have been met, the 
emphasis in the coming years will be placed on two axes: on the assessment of future-oriented quality assurance 
within the institutions and predominantly past programme results on the one hand and the promotion of a culture 
of quality and quality improvement on the other hand. This positioning calls for authoritative assessments and 
authoritative advice. In doing so, NVAO will claim a place for itself in the public debate, aimed at providing sound 
information on the one hand and thorough analyses and recommendations on the other. 

This new positioning is not formulated for the greater good and glory of NVAO. No, it fits the aspirations of the 
Flemish and Dutch governments, of the Flemish and Dutch institutions of higher education and those of their 
students, as well as those of the labour markets in both regions in order to offer the best possible higher 
education to a great diversity of students in a fast changing labour market. To all these aspirations and objectives, 
external quality assurance ought to contribute! 



Attachment 6 Internal Quality Assurance - Quality scheme per domain - an example 



Domain 12 : Policy, Strategy and Leadership 


Period : 
2010-2011 


Controller: 
L.Bollaert 


Nr Target 13 Actions for 


Indicator 15 


Resources 16 


Deadline 17 


Owner 18 


Realisation 19 



Internationalisation 



1.1 



New policy plan 
Internationalisation 



Development new policy 
plan Internationalisation 



Approval policy plan 
Internationalisation 



Time and 
discussions 



End 2010 



Working group 
Internationalisation 



Board 



I.2 



Improving 
Communication 
internally & externally 



Drawing newsletters 

Drawing newsflashes 
internally 

Extending involvement in 
Interationalisation 



Produced newsletters 

Reports on intranet per 
project 

Internal employee in x 
projects 



webstek 



intranet 



projectresources 



3x/a year (2 
in 2010) 
Every project 

2010 
2011 



Project coördinators 
Internationalisation 



LB 



Nr 



Target 



Actions for 



Indicator 



Resources 



Deadline 



Owner 



Realisation 



Cooperation NL - FL 



Associate degree 
programmes 



Embedding Associate 
degree programmes in 
the accreditation cycle 



Timeline 

Number of applications 



Subsidization 



End 2010? 



HP & RB? 



RD 



HB05-programmes 



Assessment HB05- 
opleidingen 



Pilots completed 
Protocols 

Number of applications 



FL grant 
100.000 
VL budget 



End 2010 
End 2010 
Juni 2011 



JB 



LB 



Automatic mutual 



Involved in progress and 



DFG 



FL 



2010 



GL& LB 



Board 



Domain or quality area is a defined part of the organizational process that is subject to internal quality assurance 

13 Targets are the elaborations of the strategie and operational objectives of NVAO in the current period 

14 Actions for improvement are concrete planned action that follows from the targets 



Indicators show when the target is achieved. 



Resources: like budget, time, information or required meetings 

17 Deadline is the final date of achievement 

18 Owner is the person or group responsible for the result 

19 Realisation: the finale responsibility is lodged with the board, one of its member or the managing director 



recognition degrees 
NL & VL 



preparation Decision 
Flemish Government 
(DFG) 



Involved in progress and 
preparation list of 
equivalence 



Treaty & list of 
equivalence 



FL 



2011 



Board 



AB 



More visibility of 
NVAO 



New communication plan 
To approve 



Communication plan 
3 publication Q&A? 



Own resources & 
Sponsoring 



Juni 2010 
End 2010 



KD & GL & staff 
communication 



Board 



Direct contact with 
institutions / programmes 
and quality Assurance 
agencies 



5 visites 

3 meetings 

4 Seminars / workshops 



Own & 

subsidization & 
sponsoring 



2010-04-12 
2011 



Board 

Policy advisors 



Board 



Nr 



Target 



Actions for 



Indicator 



Resources 



Deadline 



Owner 



Realisation 



New Accreditation 
System (NAS) 



NAS NL 



Preparing NAS NL 



Execution preparation 
plan 



own 



2010 



New coördinator 



Board 



Implementation NAS NL 



# institutional audit 

# programme 
accreditation 



NL subsidization 



2011 



New coördinator Policy 
advisors 



Board 



NAS FL 



Preparing NAS VL 



Preliminary draft Act 



own 



2010 



GL& LB 



Board 



Academisation FL 



Assessment programmes 



# decisions 



FL subsidization 



2010 



Policy advisors 



Board 



Nr 



Target 



IV. General strategie 

goals 

Strategie plan 



Actions for 
improvement 



Indicator 



Resources 



Deadline 



Owner 



Realisation 
Chair/Board 



Approval new strategie 
plan 



New strategie plan 



own 



End 2010 



Board 



Board 



IV.1 



System wide analysis 



Preparation of a 
structural, internal 

process 

Implementation 



Proces description 



1 new analysis 



Own 



ICT 



End 2010 



Spring 2011 



SWIFT (Information 
system) 

HB & working groep 
ICT 



Board 



Board 



IV.2 



(Initial) accreditation 



Preparation relevant 



indicators 



Subsidized 



End 2010? 



HP 



RD 







indicators 




operation 








IV.3 


Evaluation among 
stakeholders 


Extending systematic 
evaluation among 
stakeholders and follow 
up procedure 


Questionnaires 
quality criteria 


ICT 

Meetings 


End 2010? 


HP 


LB 


Nr 




Implementation 


Discussion evaluation 
results 


Subsidized 
operation 


2011 


HP 


Board 


Target 


Actions for 
improvement 


Indicator 


Resources 


Deadline 


Owner 


Realisation 
Chair/Board 


V. 


Internal operation 
core processes & IQA 


Approval new 
organisational chart and 
operation 


Approval 


Subsidized 
operation 


2010 


LK 


Board 


V.1 


Internal operation 


Approval new 
organisational chart and 
operation 


Approval 


Subsidized 
operation 


2010 


LK 


Board 


V.2 


ICT 


Further dvelopment ICT- 
architecture 


Approval contract 
award 


Subsidized 
operation 


End 2010 


GL&RD&HB& 
working groep ICT 


Board 


V.3 


Quality & Strategie 
Statement 


New Quality & Strategie 
Statement 


Approval 


Subsidized 
operation 


End 2010 


LB & HP & working 
groep IQA 


Board 



Attachment 7 Internal Quality Assurance - Report of progress per domain - an example (to be completed in Spring 2012) 



Report of progress Bi 


ELEID, STRATEGIE en LEIDERSCHAP 






Quality area 


year 


First responsible 


Document number in the Quality Manual 


Strategy, Policy and 


2010-2011 


LB in consultation with the 


To be determined 




Leadership 




Executive Board 






Global strategie 


Specif ic aims 


Results 


Documents 


Discussion 


Goals for improvement 


objectives (P) 


(P) 


(D) 


(C) 


(C) 


& Plans 2012-2013 (A) 


1. Internationalisation 


1a. New policy plan 


1 a Approved by the Executive 


1 1a. 'Actieplan 




1a. 




Internationalisation 


board on 25January 201 1 


Internationalisering 












2011' 








1b. Improving Communication 


1b. -Newsletters, including 


1 b. -NVAO newsletters 




1b 




internally & externally 


Internationalisation, were 












published and sent to NL and FL 












HE institutions. 












-On NVAO intranet activities of 












Internationalisation are reported 


-NVAO intranet, pages 










on a regular basis. 


Internationalisation 










-NVAO has improved its website 












in English 


-NVAO website in 










-Involvement of more NVAO policy 


English 










advisors in International projects 












(e.g. Croatia, Armenia, ENQA 












workshops and seminars) 









Global strategie 
objectives (P) 

1 i L 


Specific aims 
(P) 


Results 


Documents 


Discussion 
(C) 


Goals for improvement & 
Plans 201 2-201 3 (A) 


2. 

Cooperation NL - FL 


2a. Associate degree 
programmes 


2a. Assessment of 70 applications 
in 2010. 


2a. List of NVAO 
judgments in 2010. 




2a. 




2b. HB05-programmes 


2b. A pilot project completed with 
5 programmes (17 December 
2010) 


2b. Report Learning 
effects, NVAO, 
December 2010 




2b. 




2c. Automatic mutual 
recognition degrees NL & VL. 


Involved in progress and 
preparation Decision Flemish 
Government (DFG) 
Involved in progress and 
preparation list of equivalence 


2c 




2c. 




2d. More visibility of NVAO 


Approved Communication plan by 
the Executive Board, 2 June 2010 


2d. Communication plan, 
2 June 2010 




2d. 


Global strategie 


Specific aims 


Results 


Documents 


Discussion 


Goals for improvement & 












3. 

New Accreditation 
System (NAS) 


3a NAS NL 


3a. Implementation of the new 
accreditation system on 1 January 
2011 


3a. Approved new 
Frameworks (2010) 




3a. 




3b. NAS FL 


3b. Planned implementation of the 
new accreditation system per 
September 2013 


3b Current planning 
scheme, Spring 2012 




3a+b. 




3c. Academisation FL 


3c. NVAO assessed XX 
applications regarding 
Academisation in 201 and 201 1 


3c. List of NVAO 
judgments in 2010 and 
2011. 




3c. 



Global strategie 
objectives (P) 


Specific aims 
(P) 


Results 
(D) 


Documents 
(C) 


Discussion 


Goals for improvement & 
Plans 201 2-201 3 (A) 


4. 

General strategie 
goals 


4a Strateaic Dlan 


4a ADDroved bv the General 
Board on 19 February 2012 


4a NVAO strateaic Dlan 
19 th February 2012 




4a. 




4b. System wide analysis 


4b. -NVAO completed eight SWA 
in recent years. 


4b. Eight reports 




4b. 




4c. (Initial) accreditation 


Preparation relevant indicators 


4c. XX 




4c. 




4d. Evaluation among 
stakeholders 


4d. Evaluation scheme approved 
by the Executive board in April 
201 1 . This scheme is applied 
currently. 


4d. Evaluation scheme, 
April 201 1 






Global strategie 
objectives (P) 






Documents 
(C) 


Discussion 
(C) 


Goals for improvement & 
Plans 201 2-201 3 (A) 


5. 

Internal operation 

OUIc \J\ UUcoouo Cx 


5a. Internal operation 


5a. A new organizational chart is 
approved and implemented in 

Fphri iar\/ 901 1 
ncui ucii y c\j i i . 


5a. Organisational chart, 
February 201 1 




5a.. 


IQA 


5b. ICT 


5b. The new information system 
SWIT is in operation since 
January 2012 


5b. XX 




5b. 




5c. Quality & Strategie 
Statement 


5c. Approved by the General 
Board on 19 February 2012 


5c. NVAO strategie plan, 
19 ,h February 2012 







Attachment 8 Internal Quality Assurance - Evaluation scheme - April 2011 



Process 


Aim 


Target group 


Executed by 


Instrument 


Frequency and time 


Responsible for 
task 


Related SG 
ENQA and 


Institutional audit 


First feedback, 
improving approach 
and process 
institutional audit 


Level in the institution: Board 


Board and 
Procescoordinator 


Meeting: together with the 
evaluation of (initial) 
accreditation 


After the completion of the 
institutional audit. Other 
important topics can be 
discussed as well. 


Board and 
procescoordinator 




Institutional audit 


First feedback, 
improving approach 
and process 
institutional audit 
and new 
accreditation 
system 


Resonance group 


Board 


Meeting Resonance group 


Twice a year 


Board and chair 
Resonance group: 
Paul Rullmann 




Accreditation 


Improving the 
content and process 
of accreditation 


Level in the institution: 
Faculty management 


Procescoordinator 


Questionnaire (in 
consultation with the 
NVAO Board) 


Maximum once a year 
(and not to apply for 
institutions that apply for 
an institutional audit in the 
same year). A sample is 
used here 


Coördinator IQA HP 
Communication UvH 


4, 8, 9, 10, 
13, 15, 16 


Accreditation 


Reinforcement 
contact Flemish 
institutions 
Reinforcement 
image NVAO 
Improvement 
content and process 
accreditation 


Level in the institution: 
Managing directer Education and 
Quality Assurance coördinator 
(Flanders) 


1. Team (FL/NL) policy 
advisors NvS & FW, 
possibly supervised by a 
board member 

2. Team policy advisors 
and board members 


1 .Meetings structured by 
prepared evaluation 
questions 

2. Information meetings in 
Flanders 


Spring 2011 


Portfolio holder (LB) 


4, 8, 9, 10, 
13, 15, 16 


Initial accreditation 
1 


Improvement 
content and process 
initial accreditation 


Level in the institution: 
Faculty management or 
programme coördinator 


Policy advisor / 
procescoordinator 


Questionnaire (in 
consultation with the 
NVAO Board) 


After the completion of the 
process of initial 
accreditation (and not to 
apply for institutions that 
apply for an institutional 
audit in the same year). 


Coördinator IQA HP 
Communication UvH 


4, 8, 9, 10, 
13, 15, 16 


Quality of the panel 
members, student 
included 


Policy advisor / procescoordinator 


Policy advisor / 
procescoordinator 


List with attention points is 
added to the memorandum 
of the application that is 
offered to the board when 
a decision has to be taken 


After every procedure 
initial accreditation 


Coördinator IQA HP 


4, 8, 9, 10, 
13, 15, 16 



Process 


Aim 


Target group 


Executed by 


Instrument 


Frequency and time 


Responsible for 
task 


Related SG 
ENQA and 
ECA 


(Initial) accreditation 
or institutional audit 


Quality external 
secretaries 


External secretaries 


proces- 
coordinator 


List with attention points is 
added to the memorandum 
of the application that is 
offered to the board when a 
decision has to be taken 
and a evaluation form has 
to be filled in 


After every procedure 
(initial) accreditation or 
institutional audit 


Coördinator IQA HP 


4, 8, 9, 10, 13, 
15, 16 


Communication 


Image research 


Level in the institution: 
Faculty management, Head 
department, Quality Assurance 
coördinators on different levels 


Staff Communication 


Questionnaire (in 
consultation with the NVAO 
Board) 

Is part of the meeting 
between the institutional 
board and NVAO board 


Every 5 years 


Coördinator 
communication UvH 


8, 9, 10 


Communication 


Improvement internal 
communication and 
commitment to 
NVAO 


AH staf of NVAO 


Staff Communication 


Evaluation employee 
satisfaction (in consultation 
with the NVAO Board) 


Every 2 years 


Staff management and 
support UvH 


8, 9, 10 


Intemationalisation 


Improvement 
international 
processes and 
activities 


National en international structural 
partners 


Policy advisors 
Intemationalisation 


Questionnaire (in 
consultation with the NVAO 
Board) 


Once a year 


Coördinator 

Intemationalisation 

MF 


11 


Intemationalisation 


Feedback on the 
objectives, 
improvement 
processes and 
activities regarding 
the certificate 
Intemationalisation 


Level in the institution: 

Programme director and coördinator 

intemationalisation 


Policy advisors 
Intemationalisation 


Meeting 


After completion of every 
application for a certificate 
intemationalisation 


Coördinator 
Intemationalisation MF 


11 


ncocai \j\ i 1 1 laoidi o 


Onpmtinn 

v^|JCI dtlUI 1 

committees Royal 
Academy of 
Sciences, students 
included 


oiidiio lui 1 1 1 1 ii llc cü nuyai noauciiiy 

of Sciences 


Pnrtfnlin hnIHpr anrl 

responsible policy advisor 


Mpptinn with thp phnirQ nf 
iviccui ly win i ii ic oi iciii o \J\ 

the committee based on 
prepared questions 


Twn \/pac<; mpptinn with thp 
i vvu y cdi o 1 1 icc li i ly vvilii liic 

chairs of the committees 
(after the completion of 
every application round) 


Pnrtfnlin hnIHpr KD 

l \Jl LIUIIU 1 UIUCI l\L^ 

KD en FM 


4 8 9 10 11 
15, 16 


Training secretaries 


Quality training 


Secretaries that attended the 
training 


trainer 


Questionnaire 


After every training 


Coördinator IQA HP 





2 



Attachment 9 Key figures higher education the Netherlands and Flanders 



Table I: Overview ofkey figures ofhigher education in the Netherlands and Flanders (2009-2010) 



Variables / Countries / Total 


The Netherlands 


Flanders 


Total 


Inhabitants 


16.750.000 


6.250.000 


23.000.000 


Universities 


14 


6 


20 


Universities of applied sciences (NL) and 
University colleges (FL) 


41 


21 


62 


Recognised private institutions (NL) and 
Registered institutions (FL) 


60 


3 




Other statutory registered institutions (FL) 




4 




Programmes 


3500 


1400 


4900 


Fulltime students 


635.000 


195.000 


830.000 


Table II: Total enrolment higher education in the Netherlands 


Enrolment 2009-201 (x 1 000) 


Male 


Female 


Total 


Professional Education 


193,5 


209.6 


403,1 


Academie Education 


114.3 


119,0 


233,3 


Total 


304,3 


329,7 


634,0 


Table III: Evolution of enrolment in higher professional education in the Netherlands 


Evolution of Enrolment 2009-2010 (x 1000) 


Male 


Female 


Total 


2005 - 2006 


170,8 


186,0 


356,8 


2006 - 2007 


175,1 


191,6 


366,7 


2007 - 2008 


178,5 


195,8 


374,4 


2008 - 2009 


184,2 


199,5 


383,7 


2009-2010 


193,5 


209,6 


403,1 


Table IV: Evolution of enrolment in academie education in the Netherlands 


Evolution of Enrolment 2009-2010 (x 1000) 


Male 


Female 


Total 


2005 - 2006 


103,2 


102,7 


205,9 


2006 - 2007 


103,4 


105,2 


208,6 


2007 - 2008 


104,6 


108,2 


212,7 


2008 - 2009 


108,0 


112,5 


220,5 


2009-2010 


114,3 


119,0 


233,3 


Table V: Total enrolment higher education in Flanders 


Enrolment 2009-201 (x 1 000) 


Male 


Female 


Total 


Professional Education 


37,6 


52,2 


89,8 


Academie Education 


50,6 


52,7 


103,3 


Total 


88,2 


104,9 


193,1 


Table VI: Evolution of enrolment in higher professional education in Flanders 


Evolution of Enrolment 2009-2010 (x 1000) 


Male 


Female 


Total 


2005 - 2006 


46,9 


55,5 


102,4 


2006 - 2007 


46,8 


55,7 


102,5 


2007 - 2008 


47,8 


56,4 


104,2 


2008 - 2009 


50,5 


59,3 


109,8 


2009-2010 


53,7 


62,9 


116,6 



Table VII: Evolution of enrolment in academie education in Flanders 



Evolution of Enrolment 2009-2010 (x 1000) 


Male 


Female 


Total 


2005 - 2006 


26,5 


32,7 


59,2 


2006 - 2007 


27,3 


33,5 


60,8 


2007 - 2008 


28,7 


35,6 


64,3 


2008 - 2009 


32,3 


39,4 


71,7 


2009-2010 


34,6 


42,0 


76,6 



Sources: 

1 . [Structuur decreet, 4 April 2003\ ; 

2. Flemish education in figures, 2009-2010, Flemish authorities Education and Training policy area; 

3. [Jaarboek Onderwijs in cijfers CBS, 2009, 201 Oen 201 1]; 

4. www.vsnu.nl 

5. www.vlir.be ; 

6. www.vlhora.be; 

7. www.nvao.net. 



Attachment 10 Follow-up of the recommendations of the review committee 2007 
regarding the accreditation systems in the Netherlands and Flanders 

The review committee of 2007 recommended to several issues regarding the higher education system and the 
accreditation system in the Netherlands and Flanders. The recommendations concern the following topics: 

The Committee recommend further harmonization of regulations and procedures between the two countries with 
regard to: 

Legal protection of titles 

In Flanders it is forbidden by law to provide bachelor's or master's programmes or degrees in case of 
non-accredited programmes. The degrees of academie programmes have the addition Arts or Science. 
The degrees of professional bachelor's programmes have no addition. 

In the Netherlands the Veerman committee recommended on the introduction of clear titles for all 
bachelor's and master's programmes in academie and professional higher education. The 
recommendations were adopted by the Dutch Minister in his strategie statement (201 1 ). The institution 
chooses an addition to the title that fits to the profile of the programme. The programme will be 
accountable on this matter during the accreditation procedure at NVAO. When this proposal is finalized 
into law, the additions Arts and Sciences will be no longer exclusive for academie programmes. 
Professional programmes that opt for the addition Arts or Sciences must prove the appropriateness of 
these additions during the accreditation procedure. 

The Dutch minister is looking for legal sanctions against institutions that offer non-accredited bachelor's - 
and master's programmes. 

Sanctions in the case of a negative accreditation. The Flemish system (of a statutory repair period) should also be 

implemented in the Dutch system 

Since the introduction of the new accreditation system in the Netherlands on 1 January 201 1 a recovery 
period of two years has been included in the system. NVAO is authorized to grant this period in case of a 
negative assessment report (regarding accreditation), that can improve sufficiently in two years. 

The position of the macro-efficiency check 

In 2009 the Dutch Minister decided to reverse the procedure macro-efficiency check and the procedure 
initial accreditation. From then on applications for new programmes aiming at funding have to pass the 
procedure Macro-efficiency check first (at CDHO 20 ) and secondly, in case of a positive decision from the 
Minister, these programme can apply for the procedure initial accreditation at NVAO. 

The length of the accreditation cycles 

The length the accreditation cycles in the Netherlands and in Flanders differ since the start of the Dutch- 
Flemish cooperation in higher education in 2003; six years in the Netherlands, eight years in Flanders. 
NVAO has addressed this topic in the meetings of both Ministers [Comité van Ministers] in the recent 
years. It is foreseen that the third round of accreditation in Flanders (from 2021 on) will offer an 
accreditation cycle of six years. 

The scale of assessments. The outcome of the system would benefit if there were clustered assessments on a bi- 
national scale 

Clustered assessments are current practice in Flanders and in the Netherlands regarding the 
assessment of academie programmes. The assessment of professional programmes in the Netherlands 
lost the cluster approach in the first phase of the accreditation system. The government prefers, 
supported by NAVO, to reintroduce the cluster approach in professional higher education. Initiatives 
have been taken meanwhile and it has been decided that the cluster assessment will be reintroduced. 
Cluster assessments on a bi-national scale are not planned in the near future. The main objection of a 
bi-national scale approach is logistical; the number of involved programmes would be mostly too high, 
which causes a very long turnaround of the assessment process. 



CDHO [Commissie Doelmatigheid Hoger Onderwijs] is an independent advisory body of the Dutch Minister of Education, 
Culture en Sciences. 



Different time limits for the processing of applications and differing the sanctions (in case NVAO does not adhere 
to the time limit) 

In the recent years the time limits regarding the treatment of applications did not change in the 
Netherlands and Flanders. In the Netherlands the time limit is six months for an application Initial 
Accreditation, in Flanders it is four months. In the Netherlands the time limit is three months for an 
application accreditation, in Flanders it is four months. In the past years these time limits did not cause a 
lot problems, but it must be stated that NVAO could not always respect these time limits, mainly due to 
the very high workload as a consequence of several additional tasks. Since 1 st October 2009 in the 
Netherlands the Act Penalty [Wet Dwangsom] is applicable, which obliges every government 
organisation to complete applications in time. Up till now NVAO has not been punished in the cases that 
deadlines of applications have not been respected. 

In Flanders the programmes have to apply for accreditation at least two months after the public 
assessment report and six months before the end of the previous accreditation. 

The relationship between NVAO and the quality assessment agencies. There is an issue of 'low trust' that needs 
to be adjusted 

The relation between the quality assurance agencies and NVAO is influenced by the design of the 
accreditation system in the Netherlands. The assessment agencies operate on a private basis and are 
commissioned by the institutions. Furthermore, the system in the first phase lacked a recovery period, 
which sometimes caused anticipatory behaviour of the assessment panels; it seemed that negative 
jugdements were avoided to prevent the programme from serious continuity problems, because a 
negative decision would mean a shutdown of the programme. As mentioned, in the new system in the 
Netherlands a recovery period has been introduced, which is supported by all stakeholders. Moreover, in 
the new accreditation system in NL, the panels will be appointed by NVAO and act on behalf of and 
reimbursed by NVAO. The institutions will reimburse NVAO. 

In Flanders the four party platform (of VIR, VLHORA, the Flemish cabinet and NVAO) aims at reinforcing 
the relationship between NVAO and its primary stakeholders. For instance, NVAO will advice VLUHR on 
the composition of the panels. 

The further development of the system. The committee suggest looking into the concept of self-accrediting 
institutions. 

The design of the new accreditation system in the Netherlands might provide a first step towards self 
accrediting institutions, but that will be decided in about five years. The institutional audit is a new 
element of the system that offers opportunities to learn regarding the concept of institutional 
accreditation. However, the actual opinion among the stakeholders includes a preference for programme 
assessment as a necessary element of the accreditation system. Especially students are strong 
proponents of programme assessment. 

The first concepts of the new Flemish accreditation system introducé the institutional review. The 
provisional outlines of the system in the third phase offer the possibility of self accrediting institutions. 



Attachment 11 Accreditation Frameworks in the Netherlands and Flanders 
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1 Structure of the system 

Accreditation takes place at the programme level. Thus, the accreditation system continues 
to focus on the quality of individual programmes. In addition, institutions may request NVAO 
to conduct a so-called institutional quality assurance assessment. Should such a thorough 
audit reveal that an institution's quality assurance is in such good order that the quality of 
the programmes is systematically improved, wherever necessary, NVAO will then place that 
institution in a different accreditation regime. The accreditation methods practised under this 
regime differ from those implemented for programmes without a positive institutional quality 
assurance assessment. Under this regime, an assessment panel of independent experts 
assesses each programme on a limited number of standards pertaining to the essence of 
educational quality. On the basis of this assessment, NVAO decides whether or not to 
accredit that programme. This leaves the teaching staff free to devote their attention and 
energy to expert suggestions for improvement relating to the core of their teaching, rather 
than spending time on pre-conditional aspects that are better dealt with at the institutional 
level, as the trustworthiness of the institution regarding those themes has already been 
demonstrated at the institutional level. This is a system in which: 

a. institutional quality assurance assessments bolster an institution-wide internal quality 
culture; 

b. programme accreditations focus on the essence of the education provided: (improving) 
substantive quality; 

c. a proper balance is achieved between assessing programmes on the one hand and 
quality improvement on the other. 

The accreditation system comprises six assessment frameworks: 

1 . an institutional-level framework to be used for "institutional quality assurance 
assessments " 1 , the so-called institutional quality assurance assessment; 

2. a programme-level framework with "limited assessment criteria for the accreditation of 
institutions whose institutional quality assurance assessment produced a positive 
result 2 , the so-called limited programme assessment; 

3. a programme-level framework with "extensive assessment criteria for accreditations" 3 , 
the so-called extensive programme assessment (required if an institutional quality 
assurance assessment turns out negative and for institutions that have not applied for 
an institutional quality assurance assessment); 

4. a programme-level framework with "limited assessment criteria for the initial 
accreditation of new programmes provided by institutions whose institutional quality 
assurance assessment produced a positive result" 4 , the so-called limited initial 
accreditation; 

5. a programme-level framework with "extensive assessment criteria for the initial 
accreditation of new programmes" 5 , the so-called extensive initial accreditation 
(required if an institutional quality assurance assessment turns out negative and for 
institutions that have not applied for an institutional quality assurance assessment); 

6. an assessment framework to determine whether an institution or a programme has any 
distinctive features. 6 



1 Dutch Higher Education and Research Act (WHW), Articles 5a.13a - 13e. 

2 Dutch Higher Education and Research Act (WHW), Article 5a.13f. 

3 Dutch Higher Education and Research Act (WHW), Article 5a. 8. 

4 Dutch Higher Education and Research Act (WHW), Article 5a.13g. 

5 Dutch Higher Education and Research Act (WHW), Article 5a. 10a. 

6 Dutch Higher Education and Research Act (WHW), Article 5a.10. 
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The five chapters below present the first five assessment frameworks. The following is 
successively outlined for each of the assessment frameworks: its set-up, the framework 
itself, the composition of the audit panel or the assessment panel, the elements of the 
assessment process, the decisions to be taken by NVAO and the minimum documentation 
required. Chapter 7 outlines the background and the criteria that apply to the award of a 
distinctive feature. Chapter 8 defines the assessment scales that apply to (extensive and 
limited) programme assessments and presents examples for the operationalisation of said 
scales. Chapter 9 encompasses the assessment rules. 

The document at hand concludes by outlining the appeal procedures. 

Justification 

The frameworks have been formulated on the basis of the European guidelines for the 
internal quality assurance of higher education institutions. These guidelines are presented in 
Chapter 2 of the Standards and Guidelines for Quality Assurance in the European Higher 
Education Area ('European Standards and Guidelines'; ESG) of the European network for 
quality assurance agencies, the European Association for Quality Assurance (ENQA). In 
2005, this document was adopted by the education ministers during their meeting in Bergen. 
NVAO has established that the requirements with regard to institutional quality assurance 
assessment, set by the Minister of Education, Culture and Science (OCW) in his 
memorandum Focus op quality [Focus on Quality], are in excellent alignment with the ESG. 7 
NVAO has translated and rearranged the ESG in order to shift the focus from quality 
assurance to quality enhancement. 



7 

Dutch Lower Chamber, meeting year 2007-2008, 31 288 and 28 879, no. 21 . Cf. Het Hoogste Goed, Strategische agenda voor het 
hoger onderwijs-, onderzoeks- en wetenschapsbeleid [The Greatest Good, strategie agenda for higher education, research and science 
policy], Parliamentary Document 2007-2008, 31288, no. 1. 
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2 Institutional quality assurance assessment 



2.1 Set-up 

The object of the institutional quality assurance assessment is to determine whether the 
board of an institution has implemented an effective quality assurance system, based on its 
vision of the quality of the education provided, which enables it to guarantee the quality of 
the programmes offered. Institutional quality assurance assessments are not expressly 
aimed at assessing the quality of individual programmes. 

In essence, institutional quality assurance assessments revolve around five coherent 
questions: 

1 . What is the vision of the institution with regards to the quality of the education it 
provides? 

2. How does the institution intend to realise this vision? 

3. How does the institution gauge the extent to which the vision is realised? 

4. How does the institution work on improvement? 

5. Who is responsible for what? 

These five questions have been translated into five standards. Regarding each of these five 
standards, the audit panel gives a weighted and substantiated judgement on a three-point 
scale: meets, does not meet or partially meets the Standard. The audit panel subsequently 
gives a substantiated final conclusion on the question of whether an institution is in control 
with regard to the quality of its programmes. This judgement is also given on a three-point 
scale: positive, negative or conditionally positive. 
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2.2 Assessment framework for institutional quality assurance assessments 

Vision of the quality of the education provided 

Standard 1: The institution has a broadly supported vision of the quality of its education and the 
development of a quality culture. 

Expianation: This vision pertains to the institution's ambition regarding the quality of its education and its 
requirements regarding the quality of its programmes. 

For the purpose of developing a quality culture, the board of the institution encourages the 
programmes to monitor their quality and implement improvements wherever required. An 
active role by all those involved in the education provided is vitally important to this end. 

Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 



Policy 

Standard 2: The institution pursues an adequate policy in order to realise its vision of the quality of its 
education. This comprises at least: policies in the field of education, staff, facilities, 
accessibility and feasibility for students with a functional disability, embedding of research in 
the education provided, as well as the interrelation between education and the 
(international) professional field and discipline. 

Expianation: The policy fields to be assessed are not limited to those stated in the Standard but depend 
on the institution's vision of the quality of its education. Adequate policy presupposes 
concrete objectives ensuing from said vision and allocation of sufficiënt resources to 
implement said policy. 

Anchoring research in the education provided is important because all higher education 
institutions have to engage in research to some extent, even if they do not conduct research 
themselves and only wish to inform students of new scientific developments in the domain 
of the programme in which they are enrolled. This Standard expressly does not involve an 
assessment of the research itself. 

Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 



Output 

Standard 3: The institution has insight into the extent to which its vision of the quality of its education is 
realised. It gauges and evaluates the quality of its programmes on a regular basis, among 
students, staff, alumni and representatives of the professional field. 

Expianation: The institution has management information with regard to the implementation of policy and 
the output of its programmes. It also has an adequate system of internal evaluations and 
external assessments. The evaluation and gauging activities have been set up efficiently 
and provide the board of the institution with aggregated information. Institution-wide 
uniformity in the evaluation and gauging activities is not required. 



Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 
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Improvement policy 

Standard 4: The institution can demonstrate that it systematically improves the quality of its programmes 
wherever required. 

Expianation: The institution pursues an active improvement policy based on its insight into the output 
achieved. This contributes to the quality culture within the institution. 

Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 



Organisation and decision-making structure 

Standard 5: The institution has an effective organisation and decision-making structure with regard to 
the quality of its programmes, which clearly defines the tasks, authorities and 
responsibilities and which encompasses the participation of students and staff. 

Expianation: The organisation and decision-making structure enables the institution to realise its vision 
(Standard 1), its policy (Standard 2), the output (Standard 3) and its improvement policy 
(Standard 4) in a coherent fashion. 

The commitment of staff and students is demonstrated by the manner in which they are 
consulted and the consideration of their recommendations in the programmes. If laid down 
by law, the assessment of this Standard also covers the terms of reference and the 
positioning of examining boards and programme committees. 

Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 



General judgement 

Based on its vision of the quality of the education provided, the board of the institution has 
implemented an effective quality assurance system, which enables it to guarantee the 
quality of the programmes offered. 

Judgement: Positive, negative or conditionally positive (weighted and substantiated). 

If the audit panel pronounces the judgement of conditionally positive, it will explicitly state 
the relevant conditions. 
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2.3 Composition of the audit panel 

NVAO convenes and appoints an audit panel to conduct the institutional quality assurance 
assessment. The institution to be assessed is entitled to lodge substantiated objections to 
the composition of the audit panel. 

Audit panels must meet the following requirements: 

1 . the panel is composed of at least four members, including one student; 

2. the panel commands administrative, educational and audit expertise, is acquainted 
with developments in the higher education sector at home and abroad, and is 
authoritative; 

3. one of the members with administrative expertise will act as chair; 

4. the panel is independent (its members have had no ties with the institution to be 
assessed over at least the past five years). 

The audit panel is counselled by an NVAO process co-ordinator and supported by a 
secretary. The secretary and the process co-ordinator are also independent of the institution 
in question. The secretary and the process co-ordinator do not sit on the panel. 

Prior to the first visit, all panel members and the secretary certify to not maintaining any 
connections or ties with the institution in question, of either a personal or a professional 
nature, which could affect an independent judgement in either a positive or a negative 
sense, and to not having had such connections or ties with the institution during the past five 
years. 

In addition to the factual independence, as expressed above in the nature of the relationship 
and the number of years, it is essential for any prospective panel member or secretary to 
feel independent. In some cases, an independence of more than five years may not provide 
sufficiënt guarantee for an independent position; a prospective panel member or secretary 
could still experience too strong a relationship with the institution or, for example, be 
involved too closely with an institution or programme because of family ties. In such cases, 
the prospective panel member or secretary cannot sit on the panel. Panel membership 
requires a professional attitude. To that end, NVAO has formulated a code of conduct for 
panel members and secretaries. Panel members and secretaries will sign the code of 
conduct beforehand; after the assessment process, they will sign a declaration drafted by 
NVAO that the assessment has been carried out independently. 

Stakeholders such as panel members, staff or students may report to NVAO any matters 
arising during the assessment process that could affect the independence of the 
assessment. 



2.4 Assessment process 

2.4. 1 Administrative consultation 

NVAO wishes to customise its operations and take account of the diversity in organisational 
formats, the desired degree of internationalisation of the audit and the specific nature of an 
institution. For that reason, the assessment process starts off with administrative 
consultations between the institution and NVAO. These consultations focus on the 
institution's organisational structure with regard to the education it provides, the possible 
(international) composition of the audit panel, the language in which the audit must be 
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conducted, the relevant time frame and the material available in the institution for the 
purpose of the institutional quality assurance assessment. If so desired, an institution may 
use the institutional quality assurance assessment to assess the quality assurance of so- 
called non-degree programmes. 

2.4.2 Accreditation portrait 

Based on the administrative consultations with the institution, NVAO starts off by drawing up 
an "accreditation portrait". The accreditation portrait is based on the accreditation decisions 
NVAO has taken with regard to new and existing programmes during the initial stage of the 
accreditation system. Any side letters and other information available to NVAO are also 
factored in. The accreditation portrait is handed over to the audit panel. It is submitted to the 
institution in question beforehand and the institution is given an opportunity to respond. 

2.4.3 Critical reflection 

The institution draws up a critical reflection. In essence, the critical reflection answers the 
question of how the institution demonstrates that it is in control of the quality of the 
programmes offered. The critical reflection follows the standards outlined for the institutional 
quality assurance assessment framework, whereby the institution's strengths and 
weaknesses are described by reference to notable examples. The critical reflection is a self- 
contained document that can be read separately. 

The critical reflection contains a number of basic data on the institution and its programmes. 
These enable the audit panel to gain a global picture of the institution. (The required basic 
data are listed in paragraph 2.6.) 

When considering the standards, the institution itself is expected to set a course for its 
vision and policies, whereupon it is up to the audit panel to assess to what extent the 
institution manages to achieve its ambitions. This means, for example, that an audit panel 
must check whether an institution whose vision indicates that it wishes to focus on 
internationalisation of - or in - its programmes not only develops policy and makes 
resources available to that end but also evaluates and wherever necessary adjusts said 
policy. 

A similar reasoning applies to aspects such as teaching strategies, prior experiential 
learning, input from the professional field etcetera. If these have been incorporated in the 
vision and policy regarding the programmes, they should also be covered by the evaluation, 
the information gathering, an assessment of the quality achieved and measures for 
improvement, if any. 

As for staff and facilities, the institutional quality assurance assessment involves assessing 
the policy and procedures in place with regard to staff and the facilities rather than their 
programme-specific realisation. Consequently, the policy and procedures must be specified 
in the critical reflection. The implementation of policy is considered in the institutional quality 
assurance assessment but programme-specific substantive matters such as, for example, 
teachers' subject expertise are only considered in limited programme assessments. 
In its assessment, the audit panel limits itself to the institution's policy regarding the 
effectiveness of the quality assurance system in place for the programmes. The panel does 
not judge the quality of the programmes. 
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For the purpose of describing its output, an institution can include relevant public 
information, for example from the Keuzegids [Guide to Higher Education Courses] or 
Studiekeuze 123 [www.studychoice.nl]. 

The critical reflection comprises a maximum of 50 pages; it has very few appendices and 
any appendices are limited in size. (The required appendices are listed in paragraph 2.6.2.) 

2.4.4 Site visit 

The site visit for the purpose of the institutional quality assurance assessment comprises 
two components and takes a total of at least two to, in principle, five days. The panel may 
extend its visit if prompted by the circumstances. As a rule, the audit panel starts off by 
visiting the institution for a day, followed by a second visit after two to four weeks. The 
institution and the panel may decide to divert from this set-up by mutual agreement. 

First visit: exploration 

Prior to the first visit, the audit panel has studied the institution's critical reflection and the 
accreditation portrait. Prior to its first visit, the panel discusses the questions it intends to put 
to the discussion partners. During the preliminary meeting, the audit panel also discusses a 
number of documents underpinning the critical reflection. 

The first visit has an exploratory nature. The audit panel gains insight into the ins and outs 
of the institution, the specific points for attention of the board of the institution and 
satisfaction among students, teaching staff and other stakeholders. It identifies the topics to 
be investigated in more detail. 

During the first visit, the audit panel will, in any case, meet with the following discussion 
partners: 

• the board of the institution (and, if the institution so desires: a representative from the 
supervisory board); 

• the managers responsible for education; 

• quality assurance experts and other relevant staff; 

• teachers from representative bodies; 

• students from representative bodies; 

• if relevant: representatives from the professional field. 

The Schedule for the visit is drawn up by the NVAO process co-ordinator in consultation with 
the chair of the audit panel and the contact person of the institution. The panel determines 
the structure and organisation of the visit. It decides at his own discretion which teachers 
and which students it would like to see and which documents it would like to examine. In 
principle, the following pre-conditions are observed: 

• The meetings take 45 - 60 minutes. 

• In principle, the delegations of the institution comprise no more than six people. 

• In between the meetings, the audit panel takes time to deliberate. 

In addition, the panel will set aside time for open consultations. The institution and the panel 
will make these open consultations widely known, both prior to and during the visit. 

At the end of the first visit, the chair of the audit panel provides brief feedback information to 
the institution. This feedback presents the panel's first impressions of the quality assurance 
in place in the institution. In addition, it indicates the audit trails to be conducted. Audit trails 
are studies pertaining to the implementation of policy and/or the management of problems, 
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in which the audit panel follows the trail trom the institutional level to the implementation 
level or vice versa. The audit panel decides which topics to consider in the audit trails and 
which individuals it would like to see to that end. The institution may point out well-founded 
other options to the audit panel or request the panel to conduct an additional audit trail. 
In order to minimise the workload for the institution, the audit panel gives specific 
instructions regarding the documents to be studied for the audit trails and the required 
discussion partners. 

The institution prepares the second visit in consultation with the NVAO process co-ordinator. 
The panel ultimately decides on the structure and the organisation of the visit. 

Second visit: in-depth study 

During the second visit to the institution, a further discussion takes place between the audit 
panel and representatives of the institution regarding points for attention emerging from the 
meetings and the documents studied during the first visit. This discussion enables the audit 
panel to ascertain whether its initial impressions were correct. Again, the delegations of the 
institution comprise a maximum of, in principle, six persons. 

The audit panel needs to demonstrate how it has ascertained whether an institution's quality 
assurance system works. This is where the audit trails come in. There are vertical and 
horizontal audit trails. It should be noted in this regard that institutional quality assurance 
assessments expressly do not involve assessing programmes but rather assessing the 
functioning of the quality assurance system in relation to the programmes. 
In a vertical audit trail, the panel examines to what degree an element of the vision referred 
to in Standard 1 is actually put into practice in two or more programmes. All standards in the 
framework are considered in succession. For example, the panel may examine whether the 
intended international exchange of students is actually effected. 
Horizontal audit trails focus on the realisation of a single Standard (in other words: a 
component of the quality assurance system) in a number of programmes. For example, the 
functioning of programme committees or the monitoring of measures for improvement. 

At the end of the second visit, the chair of the audit panel provides brief feedback to the 
institution regarding the general judgement and the underpinning considerations. 

2.4.5 Assessment procedure within the audit panel 

The audit panel presents a judgement regarding all the standards contained in the 
assessment framework. This judgement is based on an appraisal of the positive and critical 
elements in the panel's findings. Options for the judgement are: meets, does not meet or 
partially meets the Standard. Subsequently, the panel formulates a general, weighted and 
substantiated judgement on the question of whether an institution is in control of the quality 
of the programmes it offers. That judgement is also given on a three-point scale: positive, 
negative or conditionally positive. 

2.4.6 Advisory report 

The audit panel secretary draws up an advisory report comprising 20 to 30 pages. The main 
content of the report is made up of the panel's judgements regarding the standards. It is 
important for the audit panel to include underpinnings based on the institution's critical 
reflection, the meetings with representatives of the institution and the underlying data from 
the documents studied. For that reason, the advisory report will include notable and 
representative examples. 

The advisory report is preceded by a general consideration of the judgement regarding the 
institution's quality assurance comprising a maximum of two pages. Any measures for 
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improvement will be presented in a separate paragraph of the advisory report. In addition, 
the report contains a score table with the judgements emerging trom the institutional quality 
assurance assessment, information on the dates of the site visits, the names of the 
discussion partners, a number of basic data concerning the institution (see paragraph 2.6), 
an overview of the material studied and the declarations of independence signed by the 
panel members and the secretary. In its report, the panel gives an account of the manner in 
which it has organised its visit and how it has arrived at its choice of discussion partners and 
documents. 

At the end of the second visit, the contents of the advisory report are discussed and 
tentatively laid down by the audit panel members. 

NVAO forwards the advisory report to the board of the institution once all panel members 
have approved its contents. The institution is given a term of two weeks to respond to any 
factual inaccuracies in the report, whereupon the chair of the audit panel endorses the 
report after all panel members have approved its contents. The report is signed by the chair 
and the secretary of the panel and submitted to NVAO for decision-making. If NVAO finds 
that a report raises questions or if an institution so desires, NVAO may invite the institution 
and/or the audit panel for further consultations. 



2.5 NVAO decision-making 

A "positive" judgement results in approval by NVAO for a term of six years. This means that 
the programme assessments may be conducted in accordance with the limited programme 
assessment framework. 

A "conditionally positive" judgement results in approval by NVAO under resolutive 
conditions for a term of one year. 8 During that year, the programme assessments my be 
conducted in accordance with the limited programme assessment framework. A positive 
accreditation decision or a positive initial accreditation decision based on the limited 
framework will subsequently be valid for one year. When given the "conditionally positive" 
judgement, the institution must acquire a positive judgement within a year, whereby NVAO 
ascertains whether the institution meanwhile meets the conditions set. If the institution fails 
to apply for an additional judgement or does not meet the conditions, the positive judgement 
expires. 

If within that year the institution, in the opinion of NVAO, manages to satisfy the conditions 
set on the basis of the institutional quality assurance assessment, the validity of a positive 
accreditation decision or positive initial accreditation decision will be extended to six years. 

Satisfaction of the conditions set will be assessed by an audit panel commissioned by 
NVAO. The additional assessment will basically be carried out in accordance with the 
procedure for regular institutional quality assurance assessments. The audit panel will focus 
on the conditions set earlier. 9 



Article 5a,13d, paragraph 6, Dutch Higher Education and Research Act; Accreditation Decree of the Dutch Higher Education and 
Research Act. 
9 

Article 5a,13d, paragraph 6 in conjunction with Article 5a, 13e of the Dutch Higher Education and Research Act 
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A "negative" judgement results in the withholding of approval for a minimum period of three 
years. Any programme assessments will be carried out in accordance with the extensive 
programme assessments regime. Programmes that have been accredited during the 
introduction regime based on a limited assessment or have passed initial accreditation must 
undergo additional assessment if approval is withheld following an institutional quality 
assurance assessment. The institution must apply to NVAO for such additional assessment. 
A decision regarding such applications will be taken within three months. 



2.6 Required documents 

During the assessment process, the institution will provide the audit panel with a limited 
number of documents. NVAO assumes that these are existing documents, available within 
the institution, rather than documents prepared especially for the institutional quality 
assurance assessment. The documents serve as a substantiation and if need be as 
verification. Other material is only required when explicitly requested by the panel or if the 
institution wishes to demonstrate a particular distinctive feature. 

2.6. 1 Basic data concerning the institution 

(The basic data is incorporated into the critical reflection, the advisory report and the NVAO 
decision.) 

1. Name of the institution; 

2. Status of the institution (publicly funded or legal body providing higher education); 

3. Location(s); 

4. Overview with all programmes, enrolment figures and staff numbers. 

2.6.2 Required appendices to the critical reflection 

(The list with appendices studied will be included in the advisory report.) 

1 . Mission and/or view regarding the education provided and, if available, the institution's 
latest strategie policy plan; 

2. Organisation chart. 

2.6.3 Documents for inspection during the visits 

(The list with material studied will be included in the assessment report.) 

1 . Education policy plan or similar document(s); 

2. Policy plan regarding research in relation to the programmes offered or similar 
document(s); 

3. Staff (policy) plan or similar document(s); 

4. Facilities plan or similar document(s); 

5. Policy plan regarding the accessibility and feasibility for students with a functional 
disability; 

6. Quality assurance plan; 

7. Summary and analysis of recent evaluation results and relevant management 
information; 

8. Institution's annual report on "appeals against examinations". 



Page 1 6 NVAO | Assessment frameworks for the higher education accreditation system | 22 November 201 1 



nederlands-vtaamse accreditatieorganisatie 



3 Limited programme assessment 



3.1 Set-up 

The framework for limited assessments of existing programmes is used for institutions that 
have obtained a positive judgement following an institutional quality assurance assessment. 
The assessment is based on a discussion with peers regarding the content and quality of 
the programme. It focuses on three questions: 

1 . What is the programme aiming for? 

2. How is the programme realising this aim? 

3. Is the programme achieving its objectives? 

These three questions have been translated into three standards. Regarding each of these 
three standards, an assessment panel gives a substantiated judgement on a four-point 
scale: unsatisfactory, satisfactory, good or excellent. The panel subsequently gives a 
substantiated final conclusion regarding the overall quality of the programme, on the same 
four-point scale. 
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3.2 



Standard 1: 



Explanation: 



Judgement: 



Standard 2: 



Explanation: 



Judgement: 



Standard 3: 



Explanation: 



Judgement: 



Judgement: 



Assessment framework for limited programme assessments 



Intended learning outcomes 

The intended learning outcomes of the programme have been concretised with regard to 
content, level and orientation; they meet international requirements. 

As for level and orientation (bachelor's or master's; professional or academie), the intended 
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the 
international perspective of the requirements currently set by the professional field and the 
discipline with regard to the contents of the programme. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



Teaching-learning environment 

The curriculum, staff and programme-specific services and facilities enable the incoming 
students to achieve the intended learning outcomes. 

The contents and structure of the curriculum enable the students admitted to achieve the 
intended learning outcomes. The quality of the staff and of the programme-specific services 
and facilities is essential to that end. Curriculum, staff, services and facilities constitute a 
coherent teaching-learning environment for the students. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



Assessment and achieved learning outcomes 

The programme has an adequate assessment system in place and demonstrates that the 
intended learning outcomes are achieved. 

The level achieved is demonstrated by interim and final tests, final projects and the 
performance of graduates in actual practice or in post-graduate programmes. The tests and 
assessments are valid, reliable and transparent to the students. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



General conclusion 

The quality of the programme is 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 
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The assessment is based on the following definitions. These definitions relate to both the 
scores obtained for the individual standards and the overall scores awarded to the 
programme. 

Generic quality 

The quality that can reasonably be expected in an international perspective trom a higher 
education bachelor's or master's programme. 

Unsatisfactory 

The programme does not meet the current generic quality standards and shows serious 
shortcomings in several areas. 

Satisfactory 

The programme meets the current generic quality standards and shows an acceptable level 
across its entire spectrum. 

Good 

The programme systematically surpasses the current generic quality standards across its 
entire spectrum. 

Excellent 

The programme systematically well surpasses the current generic quality standards across 
its entire spectrum and is regarded as an (inter)national example. 

Chapter 8 presents examples for the operationalisation of these assessments. 

3.3 Composition of the assessment panel 10 

It is imperative that assessment panels are composed in a manner allowing meaningful 
discussions among peers, in which the panel remains sufficiently independent. The 
institution convenes the panel, appoints a secretary and subsequently presents the panel to 
NVAO for approval. To that end the institution provides data on the expertise and 
independence of the panel members and the secretary, in a manner stipulated by NVAO. 
The institution may also commission an external quality assessment agency to convene a 
panel; in such cases the panel must also be presented to NVAO for approval. 

The panel secretary has completed NVAO training leading to certification. Every year, 
NVAO publishes a list of NVAO certified secretaries. 

Assessment panels must meet the following requirements. 

1 . The panel is composed of a minimum of four members, among whom at least two 
authoritative domain experts 11 and a student. 

2. Overall, the panel commands the following expertise: 

a. expertise regarding developments in the discipline, 

b. international expertise, 



This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of 
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out 
by the institution and a code of conduct for panel members. 

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise. 
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c. practical expertise in the professional field relevant to the programme (if 
applicable), 

d. experience in teaching and educational development at the relevant 
programme level and expertise regarding the teaching format(s) used in 
the programme 12 , 

e. student-related expertise, 

f. assessment or audit expertise. 

3. The panel is independent (its members have not had any ties with the institution 
providing the programme for at least the past five years). 

4. The panel is assisted by an independent, external secretary trained and certified by 
NVAO. The secretary does not sit on the panel. 

Prior to the visit, all panel members and the secretary certify to not maintaining any 
connections or ties with the institution in question, either as a private individual or as a 
researcher / teacher, professional or adviser, which could affect an independent judgement 
of the quality of the programme in either a positive or a negative sense, and to not having 
had such connections or ties with the institution during the past five years. 

In addition to the factual independence, as expressed above in the nature of the relationship 
and the number of years, it is essential for any panel member or secretary to feel 
independent. 

Panel members and secretaries will sign a declaration of independence and confidentiality 
prior to the assessment process. In this declaration, they attest to having taken note of the 
code of conduct. Following the assessment process, the chair and secretary sign the 
assessment report once all panel members have read and approved the report. The report 
includes a declaration that the assessment has been carried out independently. 

Stakeholders such as panel members, staff or students may report to NVAO any matters 
arising during the assessment process that could affect the independence of the 
assessment or pertain to other complaints regarding the panels or secretaries. 



3.4 Assessment process 

3.4. 1 Critical reflection 

For the purpose of the assessment by the assessment panel, the programme presents a 
critical reflection of the programme. The critical reflection should follow the standards 
outlined for the limited programme assessment framework and describes the programme's 
strengths and weaknesses. In its critical reflection, the programme outlines how it checks 
student and staff satisfaction and reports on the results. Underpinning documents are made 
available for the panel to inspect. In addition, the report indicates which measures for 
improvement have been taken following the previous assessment. The critical reflection is a 
self-contained document that can be read separately. 



This includes, for example, distance leaming, work-related courses, flexible education, skill-oriented education or education aimed at 
excellent students. 



Page 20 NVAO | Assessment frameworks for the higher education accreditation system | 22 November 201 1 



nederlands-vtaamse accreditatieorganisatie 



The assessment framework for limited programme assessments is structureel in a manner 
allowing programmes ample scope to emphasise their unique character. The programme 
may use that scope in the critical reflection. The critical reflection is the pre-eminent tooi to 
allow teachers and peers to comment on the contents of the programme. Therefore it must 
be a document in which teachers and students recognise the programme. 

In addition, the assessment framework offers opportunities to discuss the ambitions of the 
programme during the site visit, rather than focus on the results obtained in the past. What 
choices will the programme make for the future, what direction will it take? In order to 
conduct such discussions, the assessment panel is expected to be able to reflect on the 
programme's plans for the future, together with the programme's representatives. 

It is imperative that any overlap with assessments within the context of the institutional 
quality assurance assessment is avoided when drawing up the critical reflection and during 
the assessment procedure. Should any reference to institutional policy or, for example, 
departmental policy be necessary, programme assessments strictly focus on the fitness for 
purpose of the policy pursued regarding the programme in question. This does not include 
pre-conditional matters, such as the structure of quality assurance or the institution's staff 
policy; these are considered in institutional quality assurance assessments. 

The critical reflection comprises a maximum of 25 pages, excluding appendices. 

3.4.2 Site visit 

In principle, the required site visit for the purpose of a limited programme assessment takes 
one day. In the event of a collective assessment of comparable programmes within a single 
institution, the duration may be reduced proportionally. 

Prior to the visit, the assessment panel has studied a number of final projects in order to 
gain insight into the exit level attained in the programme. To that end, a selection is made 
from a comprehensive overview drawn up by the programme. The final projects, the 
relevant assessment criteria and the requirements are forwarded to the panel members 
prior to the visit, or the panel members examine the documents on site prior to the visit. 
Prior to the visit, the panel members form a preliminary opinion about the programme and 
draw up questions for their site visit. The panel factors the outcomes of the institutional 
quality assurance assessment into its judgement. 

During the site visit, the assessment panel will, in any case, meet with the programme 
management, members of the examining board and the programme committee, teachers, 
students, alumni and wherever relevant representatives of the professional field. In addition, 
the panel examines the material made available by the programme. The panel determines 
the exact scope of the discussions, the possible clustering of discussion participants and the 
further organisation of the visit. The panel decides at its own discretion which teachers and 
students it would like to see and which documents it would like to examine. In principle, the 
programme delegations comprise no more than six persons. The panel will set aside time 
for open consultations. The programme and the panel will make these open consultations 
widely known, both prior to and during the visit. In addition, the panel may visit lectures or 
other teaching-learning situations, such in consultation with the programme. 
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At the end of the site visit, the chair of the assessment panel provides brief feedback 
information to the programme regarding the general judgement and the underlying 
considerations. 

3.4.3 Assessment procedure within the assessment panel 

The assessment panel presents its judgement regarding all the standards incorporated in 
the assessment framework. This judgement is substantiated by an appraisal of the positive 
and critical elements from the panel's findings. The judgement may be: unsatisfactory, 
satisfactory, good or excellent. The panel subsequently formulates a general, weighted and 
substantiated judgement regarding the quality of the programme. This judgement is also 
given on a four-point scale, ranging from unsatisfactory to excellent. 

3.4.4 Assessment report 

The assessment panel secretary draws up an assessment report comprising some 20 
pages. The main content of the report features the panel's judgements regarding the 
standards. It is important for the audit panel to include underpinnings based on the 
programme's critical reflection, the meetings with representatives of the programme and the 
underlying data from the documents made available. The report will include significant and 
representative examples. In its report, the panel gives an account of the manner in which it 
has organised its visit and how it has arrived at its choice of discussion partners and 
documents. 

The assessment report is preceded by a summary judgement regarding the quality of the 
programme comprising a maximum of two pages. Any measures for improvement will be 
presented in a separate paragraph. In addition, the report contains a score table with the 
panel judgements, information on the date(s) of the site visit, the names of the discussion 
partners, basic data concerning the programme (see paragraph 3.6), an overview of the 
material studied and the declarations of independence signed by the panel members and 
the secretary. 

The assessment panel secretary forwards the advisory report to the board of the institution 
once all panel members have approved its contents. The institution is given the opportunity 
to respond to any factual inaccuracies in the report, whereupon the panel chair endorses the 
report after all panel members have taken note of and approved its contents. The report is 
signed by the chair and the secretary of the panel. 



3.5 NVAO decision-making 

The board of the institution applies to NVAO for accreditation based on the assessment 
report. NVAO may decide to accredit the programme, not accredit it or grant an 
improvement period. The Accreditation Decree of the Dutch Higher Education and Research 
Act stipulates how, on what grounds and under what circumstances NVAO may grant an 
improvement period. 

Because of the limited nature of the assessment, NVAO exercises more reticence in 
reviewing the assessment report than it does with extensive programme assessments. The 
positive assessment of the institution inspires sufficiënt confidence that the quality 
assurance regarding the quality of the education provided by the institution is effectively 
guaranteed. In addition, its prior approval of the assessment panels and the fact that these 
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panels' secretaries have been trained and certified provides sufficiënt context and certainty 
to resort to a more reticent review of the assessment report. The review focuses on the 
completeness and validity of the assessment report. 



3.6 Required documents 

During the assessment process, the programme provides the assessment panel with a 
limited number of documents. NVAO assumes that these are existing documents, available 
within the institution, rather than documents prepared especially for the programme 
assessment. The documents serve as a substantiation and if need be as verification. Other 
material is only required when explicitly requested by the panel or if the programme wishes 
to demonstrate a particular distinctive feature. 

3.6. 1 Basic data concerning the programme 

(The basic data is incorporated into the critical reflection, the assessment report and the 
NVAO decision.) 

Administrative data regarding the programme 

1 . Nomenclature of the programme in CROHO [central register of higher education 
programmes]; 

2. Orientation and level of the programme; 

3. Number of credits; 

4. Specialisations; 

5. Location(s); 

6. Mode (s) of study; 

7. CROHO registration number. 

Administrative data regarding the institution 

1. Name of the institution; 

2. Status of the institution (publicly funded or legal body providing higher education); 

3. Outcome of the institutional quality assurance assessment. 

Quantitative data regarding the programme 

1 . Data on intake, transfers and graduates pertaining to - if possible - the last six 
cohorts; 

2. Teacher -student ratio achieved; 

3. Average amount of face-to-face instruction per stage of the study programme (a stage 
can be expressed in, for example, regular years of study, the work placement and the 
graduation period). 

3.6.2 Required appendices to the critical reflection 

(The list of appendices studied will be incorporated into the assessment report.) 

1 . Subject-specific reference framework and the learning outcomes of the programme; 

2. Overview of the curriculum in diagram form; 

3. Outline description of the curriculum components, stating learning outcomes, 
attainment targets, teaching method(s), assessment method, literature 
(mandatory/recommended), teacher and credits; 

4. Teaching and examination regulations; 
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(Items 2 to 4 are usually reflected in a study guide, in which case this can be annexed to the 
report.) 

5. Overview of allocated staff with names, positions, scope of appointment, level and 
expertise; 

6. List of the last 25 final projects or the final projects of the past two years (or portfolios / 
projects demonstrating the exit levels attained by the students); 

7. Overview of the contacts maintained with the professional field (if relevant); 

8. Report on the institutional quality assurance assessment. 

3.6.3 Documents made available during the visit 

(The list of material studied will be incorporated into the assessment report.) 

1 . Reports on consultations in relevant committees / bodies; 

2. Test questions with corresponding assessment criteria and requirements (answer 
models) and a representative selection of actual tests administered (such as 
presentations, work placements, portfolio assessments) and assessments; 

3. Representative selection of final projects, selected by the panel, of the past two years 
with corresponding assessment criteria and requirements; 

4. Reference books and other learning materials; 

5. Summary and analysis of recent evaluation results and relevant management 
information; 

6. Documentation regarding teacher and student satisfaction. 
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4 Extensive programme assessment 



4.1 Set-up 

The framework for extensive assessments of existing programmes is used for institutions 
that have failed to obtain a positive judgement following an institutional quality assurance 
assessment. The assessment is based on a discussion with peers regarding the content 
and quality of the programme. It focuses on six questions: 

1 . What is the programme aiming for? 

2. With what curriculum? 

3. With what staff? 

4. With what services and facilities? 

5. How does the programme intend to safeguard quality? 

6. Is the programme achieving its objectives? 

These six questions have been translated into six themes and 16 standards. Regarding 
each of these standards, an assessment panel gives a substantiated judgement on a four- 
point scale: unsatisfactory, satisfactory, good or excellent. The panel subsequently gives a 
substantiated final conclusion regarding the overall quality of the programme, on the same 
four-point scale. 
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4.2 

Standard 1: 
Explanation: 

Judgement: 

Standard 2: 

Explanation: 

Judgement: 
Standard 3: 
Explanation: 

Judgement: 
Standard 4: 

Explanation: 

Judgement: 
Standard 5: 
Explanation: 
Judgement: 



Assessment framework for extensive programme assessments 



Intended learning outcomes 

The intended learning outcomes of the programme have been concretised with regard to 
content, level and orientation; they meet international requirements. 

As for level and orientation (bachelor's or master's; professional or academie), the intended 
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the 
international perspective of the requirements currently set by the professional field and the 
discipline with regard to the contents of the programme. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



Curriculum 

The orientation of the curriculum assures the development of skills in the field of scientific 
research and/or the professional practice. 

The curriculum has demonstrable links with current developments in the professional field 
and the discipline. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The contents of the curriculum enable students to achieve the intended learning outcomes. 

The learning outcomes have been adequately translated into attainment targets for 
(components of) the curriculum. Students follow a study curriculum which is coherent in 
terms of content. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The structure of the curriculum encourages study and enables students to achieve the 
intended learning outcomes. 

The teaching concept is in line with the intended learning outcomes and the teaching 
formats tie in with the teaching concept. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The curriculum ties in with the qualifications of the incoming students. 

The admission requirements are realistic with a view to the intended learning outcomes. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 
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Standard 6: 
Explanation: 

Judgement: 
Standard 7: 

Explanation: 

Judgement: 

Standard 8: 
Explanation: 

Judgement: 
Standard 9: 

Explanation: 

Judgement: 
Standard 10: 
Judgement: 

Standard 11: 

Judgement: 
Standard 12: 



The curriculum is feasible. 

Factors pertaining to the curriculum and hindering students' progress are removed as far as 
possible. In addition, students with functional disabilities receive additional career tutoring. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The programme meets statutory requirements regarding the scope and duration of the 
curriculum. 

Scope and duration: 

Bachelor's programmes (professional orientation): 240 credits; 
Bachelor's programmes (academie orientation): in principle, a minimum of 180 credits; 
Master's programmes (professional orientation): in principle, a minimum of 60 credits; 
Master's programmes (academie orientation): in principle, a minimum of 60 credits, 
depending on the programme. 

Unsatisfactory, satisfactory (weighted and substantiated). 



Staff 

The programme has an effective staff policy in place. 

The staff policy provides for the qualifications, training, assessment and size of the staff 
required for the realisation of the curriculum. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The staff is qualified for the realisation of the curriculum in terms of content, educational 
expertise and organisation. 

The factual expertise available among the staff ties in with the requirements set for 
professional or academie higher education programmes. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The size of the staff is sufficiënt for the realisation of the curriculum. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



Services and facilities 

777e accommodation and the facilities (infrastructure) are sufficiënt for the realisation of the 
curriculum. 

Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

Tutoring and student Information provision bolster students' progress and tie in with the 
needs of students. 
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Judgement: Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



Quality assurance 

Standard 13: The programme is evaluated on a regular basis, partly on the basis of assessable targets. 

Expianation: The programme ensures the quality of the intended learning outcomes, the curriculum, the 
staff, the services and facilities, the assessments and the learning outcomes achieved 
through regular evaluations. The programme also collects management information 
regarding the success rates and the staff-student ratio. 

Judgement: Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

Standard 14: The outcomes of these evaluations constitute the basis for demonstrable measures for 
improvement that contribute to the realisation of the targets. 

Judgement: Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

Standard 15: Programme committees, examining boards, staff, students, alumni and the relevant 

professional field of the programme are actively involved in the programme's internat quality 
assurance. 

Judgement: Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 



Assessment and learning outcomes achieved 

Standard 16: The programme has an adequate assessment system in place and demonstrates that the 
intended learning outcomes are achieved. 

Expianation: The level achieved is demonstrated by interim and final tests, final projects and the 

performance of graduates in actual practice or in subsequent programmes. The tests and 
assessments are valid, reliable and transparent to the students. 

Judgement: Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 
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General conclusion 

The quality of the programme is 

Judgemenu Unsatisfactory, satisfactory, good, excellent (weighted and substantiated). 

The assessment is based on the following definitions. These definitions relate to both the 
scores obtained for the individual standards and the overall scores awarded to the 
programme. 

Generic quality 

The quality that can reasonably be expected in an international perspective from a higher 
education bachelor's or master's programme. 

Unsatisfactory 

The programme does not satisfy the current generic quality standards and shows serious 
shortcomings in several areas. 

Satisfactory 

The programme satisfies the current generic quality standards and shows an acceptable 
level across its entire spectrum. 

Good 

The programme systematically surpasses the current generic quality standards across its 
entire spectrum. 

Excellent 

The programme systematically well surpasses the current generic quality standards across 
its entire spectrum and is regarded as an (inter)national example. 

Chapter 8 presents examples for the operationalisation of these assessments. 

4.3 Composition of the assessment panel 13 

It is imperative that assessment panels are composed in a manner allowing meaningful 
discussions among peers, in which the panel remains sufficiently independent. The 
institution convenes the panel, appoints a secretary and subsequently presents the panel to 
NVAO for approval. To that end the institution provides data on the expertise and 
independence of the panel members and the secretary, in a manner stipulated by NVAO. 
The institution may also commission an external quality assessment agency to convene a 
panel; in such cases the panel must also be presented to NVAO for approval. 

The panel secretary has completed NVAO training leading to certification. Every year, 
NVAO publishes a list of NVAO certified secretaries. 

Assessment panels must meet the following requirements. 



This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of 
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out 
by the institution and a code of conduct for panel members. 
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1 . The panel is composed of a minimum of four members, among whom at least two 
authoritative domain experts 14 and a student. 



2. Overall, the panel commands the following expertise: 

a. expertise regarding developments in the discipline, 

b. international expertise, 

c. practical expertise in the professional field relevant to the programme (if 
applicable), 

d. experience in teaching and educational development at the relevant 
programme level and expertise regarding the teaching format(s) used in 
the programme 15 , 

e. student-related expertise, 

f. assessment or audit expertise. 



3. The panel is independent (its members have not had any ties with the institution 
providing the programme for at least the past five years). 

4. The panel is assisted by an independent, external secretary trained and certified by 
NVAO. The secretary does not sit on the panel. 

Prior to the visit, all panel members and the secretary certify to not maintaining any 
connections or ties with the institution in question, either as a private individual or as a 
researcher / teacher, professional or adviser, which could affect an independent judgement 
of the quality of the programme in either a positive or a negative sense, and to not having 
had such connections or ties with the institution during the past five years. 

In addition to the factual independence, as expressed above in the nature of the relation and 
the number of years, it is essential for any panel member or secretary to feel independent. 
In some cases, an independence of more than five years may not provide sufficiënt 
guarantee for an independent position; a prospective panel member or secretary could still 
experience too strong a relationship with the institution or be involved too closely with an 
institution or programme, for example because of family ties. In such cases, the prospective 
panel member or secretary cannot sit on the panel. Panel membership requires a 
professional attitude. To that end, NVAO has formulated a code of conduct for panel 
members and secretaries. This code of conduct comprises elements pertaining to the 
independence, confidentiality and attitude of the panel members and the secretary during 
the assessment process. 

Panel members and secretaries will sign a declaration of independence and confidentiality 
prior to the assessment process. In this declaration, they attest to having taken note of the 
code of conduct. Following the assessment process, the chair and secretary sign the 
assessment report once all panel members have read and approved the report. The report 
includes a declaration that the assessment has been carried out independently. 



14 

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise. 

15 

This includes, for example, distance learning, work-related courses, flexible education, skill-oriented education or education aimed at 
excellent students. 
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Stakeholders such as panel members, staff or students may report to NVAO any matters 
arising during the assessment process that could affect the independence of the 
assessment or pertain to other complaints regarding the panels or secretaries. 

4.4 Assessment process 

4.4. 1 Critical reflection 

For the purpose of the assessment by the assessment panel, the programme presents a 
critical reflection of the programme. The critical reflection follows the standards outlined for 
the limited programme assessment framework and describes the programme's strengths 
and weaknesses. In its critical reflection, the programme outlines how it checks student and 
staff satisfaction and reports on the results. Underpinning documents are made available for 
the panel to inspect. In addition, the report indicates which measures for improvement have 
been taken following the previous assessment. The critical reflection is a self-contained 
document that can be read separately. 

The critical reflection is the pre-eminent tooi to allow teachers and peers to comment on the 
contents of the programme. Therefore, it must be a document in which teachers and 
students recognise the programme. 

In addition, the assessment framework offers opportunities to discuss the ambitions of the 
programme during the site visit, rather than focus on the results obtained in the past. What 
choices will the programme make for the future, what direction will it take? In order to 
conduct such discussions, the assessment panel is expected to be able to reflect on the 
programme's plans for the future, together with the programme's representatives. 

The critical reflection comprises a maximum of 40 pages, excluding appendices. 

4.4.2 Site visit 

The required site visit for the purpose of an extensive programme assessment takes about 
two days. In the event of a collective assessment of comparable programmes within a single 
institution, the duration may be reduced proportionally. 

Prior to the visit, the assessment panel has studied a number of final projects in order to 
gain insight into the exit level attained in the programme. To that end, a selection is made 
from a comprehensive overview drawn up by the programme. The final projects, the 
relevant assessment criteria and the requirements are forwarded to the panel members 
prior to the visit, or the panel members examine the documents on site prior to the visit. 
Prior to the visit, the panel members form a preliminary opinion about the programme and 
draw up questions for their site visit. 

During the site visit, the assessment panel will, in any case, meet with the programme 
management, members of the examining board and the programme committee, teachers, 
students, alumni and wherever relevant, representatives of the professional field. In 
addition, the panel examines the material made available by the programme. The panel 
determines the exact scope of the discussions, the possible clustering of discussion 
participants and the further organisation of the visit. The panel decides at its own discretion 
which teachers and students it would like to see and which documents it would like to 
examine. In principle, the programme delegations comprise no more than six persons. The 
panel will set aside time for open consultations. The programme and the panel will make 
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these open consultations widely known, both prior to and during the visit. In addition, the 
panel may visit lectures or other teaching-learning situations, such in consultation with the 
programme. 

At the end of the site visit, the chair of the assessment panel provides brief feedback 
information to the programme regarding the general judgement and the underlying 
considerations. 

4.4.3 Assessment procedure within the assessment panel 

The assessment panel presents its judgement regarding all the standards incorporated in 
the assessment framework. This judgement is substantiated by an appraisal of the positive 
and critical elements from the panel's findings. The judgement may be: unsatisfactory, 
satisfactory, good or excellent. The panel subsequently formulates a general, weighted and 
substantiated judgement regarding the quality of the programme. This judgement is also 
given on a four-point scale, ranging from unsatisfactory to excellent. 

4.4.4 Assessment report 

The assessment panel secretary draws up an assessment report comprising some 30 
pages. The main content of the report features the panel's judgements regarding the 
standards. It is important for the audit panel to include underpinnings based on the 
programme's critical reflection, the meetings with representatives of the programme and the 
underlying data from the documents made available. The report will include significant and 
representative examples. In its report, the panel gives an account of the manner in which it 
has organised its visit and how it has arrived at its choice of discussion partners and 
documents. 

The assessment report is preceded by a summary judgement regarding the quality of the 
programme, comprising a maximum of two pages. Any measures for improvement will be 
presented in a separate paragraph. In addition, the report contains a score table with the 
panel judgements, information on the date(s) of the site visit, the names of the discussion 
partners, basic data concerning the programme (see paragraph 4.6), an overview of the 
material studied and the declarations of independence signed by the panel members and 
the secretary. 

The assessment panel secretary forwards the advisory report to the board of the institution 
once all panel members have approved its contents. The institution is given the opportunity 
to respond to any factual inaccuracies in the report, whereupon the panel chair endorses the 
report after all panel members have taken note of and approved its contents. The report is 
signed by the chair and the secretary of the panel. 



4.5 NVAO decision-making 

The board of the institution applies to NVAO for accreditation based on the assessment 
report. NVAO may decide to accredit the programme, not accredit it or grant an 
improvement period. The Accreditation Decree of the Dutch Higher Education and Research 
Act stipulates how, on what grounds and under what circumstances NVAO may grant an 
improvement period. 
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4.6 Required documents 

During the assessment process, the programme provides the assessment panel with a 
limited number of documents. NVAO assumes that these are existing documents, available 
within the institution, rather than documents prepared especially tor the programme 
assessment. The documents serve as a substantiation and if need be as verification. Other 
material is only required when explicitly requested by the panel or if the programme wishes 
to demonstrate a particular distinctive feature. 

4.6. 1 Basic data concerning the programme 

(The basic data is incorporated into the critical reflection, the assessment report and the 
NVAO decision.) 

Administrative data regarding the programme 

1 . Nomenclature of the programme in CROHO [central register of higher education 
programmes]; 

2. Orientation and level of the programme; 

3. Number of credits; 

4. Specialisations; 

5. Location(s); 

6. Mode (s) of study; 

7. CROHO registration number. 

Administrative data regarding the institution 

1 . Name of the institution; 

2. Status of the institution (publicly funded or legal body providing higher education); 
Quantitative data regarding the programme 

1 . Data on intake, transfers and graduates pertaining to - if possible - the last six cohorts; 

2. Teacher -student ratio achieved; 

3. Average amount of face-to-face instruction per stage of the study programme (a stage 
can be expressed in, for example, regular years of study, the work placement and the 
graduation period). 

4.6.2 Required appendices to the critical reflection 

(The list of appendices studied will be incorporated into the assessment report.) 

1 . Subject-specific reference framework and the learning outcomes of the programme; 

2. Overview of the curriculum in diagram form; 

3. Outline description of the curriculum components, stating learning outcomes, 
attainment targets, teaching method(s), assessment method, literature 
(mandatory/recommended), teacher and credits; 

4. Teaching and examination regulations; 

(Items 2 to 4 are usually reflected in a study guide, in which case this can be annexed to the 
report.) 

5. Overview of allocated staff with names, positions, scope of appointment, level and 
expertise; 

6. List of the last 25 final projects or the final projects of the past two years (or portfolios / 
projects demonstrating the exit levels attained by the students); 
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7. Overview of the contacts maintained with the professional field (if relevant). 

4.6.3 Documents made available during the visit 

(The list of material studied will be incorporated into the assessment report.) 

1 . Education policy plan or similar document(s); 

2. Policy plan regarding research in relation to the programmes offered or similar 
document(s); 

3. Staff (policy) plan or similar document(s); 

4. Services and facilities plan or similar document(s); 

5. Quality assurance plan; 

6. Policy plan regarding the accessibility and feasibility of the programme for students with 
a functional disability; 

7. Summary and analysis of recent evaluation results and relevant management 
information; 

8. Documentation regarding student and staff satisfaction; 

9. Reports on consultations in relevant committees / bodies; 

10. Test questions with corresponding assessment criteria and requirements (answer 
models) and a representative selection of actual tests administered (such as 
presentations, work placements, portfolio assessments) and assessments; 

1 1 . Representative selection of final projects, selected by the panel, of the past two years 
with corresponding assessment criteria and requirements; 

12. Reference books and other learning materials. 



Page 34 NVAO | Assessment frameworks for the higher education accreditation system | 22 November 201 1 



nederlands-vtaamse accreditatieorganisatie 



5 Limited initial accreditation 



5.1 Set-up 

The framework for limited assessments of new programmes is used for institutions that have 
obtained a positive judgement following an institutional quality assurance assessment. The 
assessment is based on a discussion with peers regarding the content and quality of the 
programme. It focuses on four questions: 

1. What is the programme aiming for? 

2. How does the programme intend to achieve its objectives? 

3. How does the programme intend to assess its performance? 

4. Does the programme have sufficiënt financial resources? 

These four questions have been translated into four standards. Regarding each of these 
standards, an assessment panel gives a substantiated judgement on a two-point scale: 
unsatisfactory or satisfactory. The panel subsequently gives a substantiated final conclusion 
regarding the quality of the programme, on the same two-point scale. 
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5.2 Assessment framework for limited initial accreditations 



Intended learning outcomes 

Standard 1: The intended learning outcomes of the programme have been concretised with regard to 
content, level and orientation; they meet international requirements. 

Expianation: As for level and orientation (bachelor's or master's; professional or academie), the intended 
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the 
international perspective of the requirements currently set by the professional field and the 
discipline with regard to the contents of the programme. 



Judgement: Unsatisfactory, satisfactory (weighted and substantiated). 



Teaching-learning environment 

Standard 2: The curriculum, staff and programme-specific services and facilities enable incoming 
students to achieve the intended learning outcomes. 

Expianation: The contents and structure of the curriculum enable the students admitted to achieve the 

intended learning outcomes. The quality of the staff and of the programme-specific services 
and facilities is essential to that end. Curriculum, staff, services and facilities constitute a 
coherent teaching-learning environment for the students. 



Judgement: Unsatisfactory, satisfactory (weighted and substantiated). 



Assessment 

Standard 3: The programme has an adequate assessment system in place. 

Expianation: The level achieved is demonstrated by interim and final tests. The tests and assessments 
are valid, reliable and transparent to the students. 

Judgement: Unsatisfactory, satisfactory (weighted and substantiated). 



Graduation guarantee and financial provisions 

Standard 4: The institution guarantees students that they can complete the entire curriculum and makes 
sufficiënt financial provisions available. 

Expianation: The graduation guarantee spans a reasonable period of time that is related to the length of 
the studies. 



Judgement: Unsatisfactory, satisfactory (weighted and substantiated). 
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General conclusion 

The quality of the programme is 
Judgemenu Unsatisfactory, satisfactory (weighted and substantiated). 



The assessment is based on the following definitions. These definitions relate to both the 
scores obtained for the individual standards and the overall scores awarded to the 
programme. 

Generic quality 

The quality that can reasonably be expected in an international perspective trom a higher 
education bachelor's or master's programme. 

Unsatisfactory 

The programme does not satisfy the generic quality standards. 
Satisfactory 

The programme satisfies the generic quality standards. 

For programmes that are not entirely new and programmes that are being converted, the 
learning outcomes achieved will be factored into the judgement. 

5.3 Composition of the assessment panel 16 

NVAO convenes and appoints the assessment panel that will conduct the initial 
accreditation. The programme to be assessed is entitled to lodge substantiated objections to 
the composition of the assessment panel. 

It is imperative that assessment panels are composed in a manner allowing meaningful 
discussions among peers, in which the panel remains sufficiently independent. 

The panel secretary has completed NVAO training leading to certification. Every year, 
NVAO publishes a list of NVAO certified secretaries. 

Assessment panels must meet the following requirements. 

1 . The panel is composed of a minimum of four members, among whom at least two 
authoritative domain experts 17 and a student. 

2. Overall, the panel commands the following expertise: 

a. expertise regarding developments in the discipline, 

b. international expertise, 

c. practical expertise in the professional field relevant to the programme (if 
applicable), 



This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenstelling [Requirements for the composition of 
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out 
byhe institution and a code of conduct for panel members. 

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise. 
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d. experience in teaching and educational development at the relevant 
programme level and expertise regarding the teaching format(s) used in 
the programme 18 , 

e. student-related expertise, 

f. assessment or audit expertise. 

3. The panel is independent (its members have not had any ties with the institution 
providing the programme tor at least the past five years). 

4. The panel is assisted by an independent, external secretary trained and certified by 
NVAO. The secretary does not sit on the panel. 

The assessment panel is counselled by an NVAO process co-ordinator and supported by a 
secretary. The secretary and the process co-ordinator are also independent of the institution 
in question. The secretary and the process co-ordinator do not sit on the panel. 

Prior to the visit, all panel members and the secretary certify to not maintaining any 
connections or ties with the institution in question, either as a private individual or as a 
researcher / teacher, professional or adviser, which could affect an independent judgement 
of the quality of the programme in either a positive or a negative sense, and to not having 
had such connections or ties with the institution during the past five years. 
In addition to the factual independence, as expressed above in the nature of the relation and 
the number of years, it is essential for any panel member or secretary to feel independent. 
In some cases, an independence of more than five years may not provide sufficiënt 
guarantee for an independent position; a prospective panel member or secretary could still 
experience too strong a relationship with the institution or be involved too closely with an 
institution or programme, for example because of family ties. In such cases, the prospective 
panel member or secretary cannot sit on the panel. Panel membership requires a 
professional attitude. To that end, NVAO has formulated a code of conduct for panel 
members and secretaries. This code of conduct comprises elements pertaining to the 
independence, confidentiality and attitude of the panel members and the secretary during 
the assessment process. 

Panel members and secretaries will sign a declaration of independence and confidentiality 
prior to the assessment process. In this declaration, they attest to having taken note of the 
code of conduct. Following the assessment process, the chair and secretary sign the 
assessment report once all panel members have read and approved the report. The report 
includes a declaration that the assessment has been carried out independently. 

Stakeholders such as panel members, staff or students may report to NVAO any matters 
arising during the assessment process that could affect the independence of the 
assessment. 



This includes, for example, distance leaming, work-related courses, flexible education, skill-oriented education or education aimed at 
excellent students. 
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5.4 Assessment process 

5.4. 1 Information dossier 

For the purpose of the assessment by the assessment panel, the programme presents an 
information dossier regarding the programme. The information dossier should follow the 
standards outlined for the limited initial accreditation framework. It positions the programme 
in relation to existing (and new) programmes at home and abroad. The critical reflection is a 
self-contained document that can be read separately. 

The framework for limited initial accreditations is structured in a manner allowing 
programmes ample scope to emphasise their unique character. The programme may use 
that scope in its information dossier. The information dossier is the pre-eminent tooi to allow 
teachers and peers to comment on the contents of the programme. It must be a document 
reflecting the commitment of the stakeholders. 

It is imperative that any overlap with assessments within the context of the institutional 
quality assurance assessment is avoided when drawing up the information dossier and 
during the assessment procedure. Should any reference to institutional policy or, for 
example, departmental policy be necessary, programme assessments strictly focus on the 
fitness for purpose of the policy pursued regarding the programme in question. This does 
not include pre-conditional matters, such as the structure of quality assurance or the 
institution's staff policy; these are considered in institutional quality assurance assessments. 

The information dossier comprises a maximum of 20 pages, excluding appendices. 

5.4.2 Site visit 

In principle, the required site visit for the purpose of limited initial accreditations takes one 
day. Prior to the visit, the panel members will have formed a preliminary opinion about the 
programme and drawn up questions for their site visit. The panel factors the outcomes of 
the institutional quality assurance assessment into its judgement. 

During the site visit, the assessment panel meets with the (prospective) programme 
management, the (prospective) members of the examining board and the programme 
committee, (prospective) teachers and, wherever relevant, representatives of the 
professional field. In addition, the panel examines the material made available by the 
programme. The panel determines the exact scope of the discussions, the possible 
clustering of discussion participants and the further organisation of the visit. The panel 
decides at its own discretion which teachers and students it would like to see and which 
documents it would like to examine. In principle, the programme delegations comprise no 
more than six persons. 

At the end of the site visit, the chair of the assessment panel provides brief feedback 
information to the programme regarding the general judgement and the underlying 
considerations. 

5.4.3 Assessment procedure within the assessment panel 

The assessment panel presents its judgement regarding all the standards incorporated in 
the assessment framework. This judgement is substantiated by an appraisal of the positive 
and critical elements from the panel's findings. The judgement may be: unsatisfactory or 
satisfactory. The panel subsequently formulates a general, weighted and substantiated 
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judgement regarding the quality of the programme. This judgement is given on the same 
two-point scale (unsatisfactory or satisfactory). 

5.4.4 Advisory report 

The assessment panel secretary draws up an advisory report comprising some 20 pages. 
The main content of the report is made up of the panel's judgements regarding the 
standards, including underpinnings based on the programme's information dossier, the 
meetings with representatives of the programme and the underlying data from the 
documents studied. The report will include significant and representative examples. In the 
report, the panel gives an account of the manner in which it has organised its visit and how 
it has arrived at its choice of discussion partners and documents. 

The assessment report is preceded by a summary judgement regarding the quality of the 
programme comprising a maximum of two pages. Any measures for improvement will be 
presented in a separate paragraph. In addition, the report contains a score table with the 
panel judgements, information on the date(s) of the site visit, the names of the discussion 
partners, basic data concerning the programme (see paragraph 5.6), an overview of the 
material studied and the declarations of independence signed by the panel members and 
the secretary. 

NVAO forwards the advisory report to the board of the institution once all panel members 
have approved its contents. The institution is given a term of two weeks to respond to any 
factual inaccuracies in the report, whereupon the panel chair endorses the report after all 
panel members have taken note of and approved its contents. The report is signed by the 
chair and the secretary of the panel and submitted to NVAO for decision-making. If NVAO 
finds that a report raises questions or if an institution so desires, NVAO may invite the 
programme and/or the assessment panel for further consultations. 



5.5 NVAO decision-making 

Basically, NVAO can take two decisions: a positive initial accreditation decision for a period 
of six years, or a negative initial accreditation decision. 

In special cases, NVAO may attach conditions to its decision. In that case, the programme 
must apply for additional assessment within a year, whereupon NVAO ascertains whether 
the programme meanwhile meets the conditions set. If the programme fails to apply for an 
additional assessment or does not meet the conditions, the positive decision expires. 
Satisfaction of the conditions set will be assessed by an assessment panel commissioned 
by NVAO. The additional assessment will basically be carried out in accordance with the 
procedure for regular limited initial accreditations. The assessment panel will focus on the 
programme's shortcomings identified earlier. 



5.6 Required documents 

During the assessment process, the programme provides the assessment panel with a 
limited number of documents. NVAO assumes that these are existing documents, available 
within the programme or the institution, rather than documents prepared especially for the 
programme assessment. The documents serve as a substantiation and if need be as 
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verification. Other material is only required when explicitly requested by the panel or if the 
programme wishes to demonstrate a particular distinctive feature. 

5.6. 1 Basic data concerning the programme 

(The basic data is incorporated into the information dossier, the advisory report and the 
NVAO decision.) 

Administrative data regarding the programme 

1 . Nomenclature of the programme; 

2. Orientation and level of the programme; 

3. Number of credits; 

4. Specialisations; 

5. Location(s); 

6. Mode (s) of study; 

Administrative data regarding the institution 

1 . Name of the institution; 

2. Status of the institution (publicly funded or legal body providing higher education); 

3. Outcome of the institutional quality assurance assessment. 

Quantitative data regarding the programme 

1 . Intended teacher-student ratio; 

2. Intended amount of face-to-face instruction per stage of the study programme (a stage 
can be expressed in, for example, regular years of study, the work placement and the 
graduation period). 

5.6.2 Required appendices to the information dossier 

(The list of appendices studied will be incorporated into the advisory report.) 

1 . Subject-specific reference framework and the learning outcomes of the programme; 

2. Overview of the curriculum in diagram form; 

3. Outline description of the curriculum components for the first year, stating learning 
outcomes, attainment targets, teaching method(s), assessment method, literature 
(mandatory/recommended), teacher and credits; 

4. Teaching and examination regulations; 

5. Overview of allocated staff with names, positions, scope of appointment, level and 
expertise; 

6. If so required, the macro-efficiency decision; 

7. Overview of the contacts maintained with the professional field (if relevant); 

8. Report on the institutional quality assurance assessment. 

5.6.3 Documents made available during the visit 

(The list of material studied will be incorporated into the advisory report.) 

1 . Reports on consultations in relevant committees / bodies; 

2. Reference books and other learning materials. 
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6 Extensive initial accreditation 



6.1 Set-up 

The framework for extensive assessments of new programmes (extensive initial 
accreditation framework) is used for institutions that have failed to obtain a positive 
judgement following an institutional quality assurance assessment. The assessment is 
based on a discussion with peers regarding the content and quality of the programme. It 
focuses on seven questions: 

1. What is the programme aiming for? 

2. With what curriculum? 

3. With what staff? 

4. With what services and facilities? 

5. How does the programme intend to safeguard quality? 

6. How does the programme intend to assess its performance? 

7. Does the programme have sufficiënt financial resources? 

These seven questions have been translated into seven themes and 16 standards. 
Regarding each of these standards, an assessment panel gives a substantiated judgement 
on a two-point scale: unsatisfactory or satisfactory. The panel subsequently gives a 
substantiated final conclusion regarding the overall quality of the programme, on the same 
two-point scale. 
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6.2 

Standard 1: 
Explanation: 

Judgement: 

Standard 2: 

Explanation: 

Judgement: 
Standard 3: 
Explanation: 

Judgement: 
Standard 4: 

Explanation: 

Judgement: 
Standard 5: 
Explanation: 
Judgement: 



Assessment framework for extensive initial accreditations 



Intended learning outcomes 

The intended learning outcomes of the program me have been concretised with regard to 
content, level and orientation; they meet international requirements. 

As for level and orientation (bachelor's or master's; professional or academie), the intended 
learning outcomes fit into the Dutch qualifications framework. In addition, they tie in with the 
international perspective of the requirements currently set by the professional field and the 
discipline with regard to the contents of the programme. 

Unsatisfactory, satisfactory (weighted and substantiated). 



Curriculum 

The orientation of the curriculum assures the development of skills in the field of scientific 
research and/or the professional practice. 

The curriculum has demonstrable links with current developments in the professional field 
and the discipline. 

Unsatisfactory, satisfactory (weighted and substantiated). 

The contents of the curriculum enable students to achieve the intended learning outcomes. 

The learning outcomes have been adequately translated into attainment targets for 
(components of) the curriculum. Students follow a study curriculum which is coherent in 
terms of content. 

Unsatisfactory, satisfactory (weighted and substantiated). 

The structure of the curriculum encourages study and enables students to achieve the 
intended learning outcomes. 

The teaching concept is in line with the intended learning outcomes and the teaching 
formats tie in with the teaching concept. 

Unsatisfactory, satisfactory (weighted and substantiated). 

The curriculum ties in with the qualifications of the incoming students. 

The admission requirements are realistic with a view to the intended learning outcomes. 

Unsatisfactory, satisfactory (weighted and substantiated). 
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Standard 6: 
Explanation: 

Judgement: 
Standard 7: 

Explanation: 

Judgement: 

Standard 8: 
Explanation: 

Judgement: 
Standard 9: 

Explanation: 

Judgement: 
Standard 10: 
Judgement: 

Standard 11: 

Judgement: 
Standard 12: 



The curriculum is feasible. 

Factors pertaining to the curriculum and hindering students' progress are removed as far as 
possible. In addition, students with functional disabilities receive additional career tutoring. 

Unsatisfactory, satisfactory (weighted and substantiated). 

The programme meets statutory requirements regarding the scope and duration of the 
curriculum. 

Scope and duration: 

Bachelor's programmes (professional orientation): 240 credits; 
Bachelor's programmes (academie orientation): in principle, a minimum of 180 credits; 
Master's programmes (professional orientation): in principle, a minimum of 60 credits; 
Master's programmes (academie orientation): in principle, a minimum of 60 credits, 
depending on the programme. 

Unsatisfactory, satisfactory (weighted and substantiated). 



Staff 

The programme has an effective staff policy in place. 

The staff policy provides for the qualifications, training, assessment and size of the staff 
required for the realisation of the curriculum. 

Unsatisfactory, satisfactory (weighted and substantiated). 

The staff is qualified for the realisation of the curriculum in terms of content, educational 
expertise and organisation. 

The factual expertise available among the staff ties in with the requirements set for 
professional or academie higher education programmes. 

Unsatisfactory, satisfactory (weighted and substantiated). 

The size of the staff is sufficiënt for the realisation of the curriculum. 

Unsatisfactory, satisfactory (weighted and substantiated). 



Services and facilities 

777e accommodation and the facilities (infrastructure) are sufficiënt for the realisation of the 
curriculum. 

Unsatisfactory, satisfactory (weighted and substantiated). 

Tutoring and student Information provision bolster students' progress and tie in with the 
needs of students. 
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Judgement: 

Standard 13: 
Explanation: 

Judgement: 
Standard 14: 

Judgement: 

Standard 15: 
Explanation: 

Judgement: 

Standard 16: 
Explanation: 
Judgement: 

Judgement: 



Unsatisfactory, satisfactory (weighted and substantiated). 



Quality assurance 

The programme is evaluated on a regular basis, partly on the basis of assessable targets. 

The programme ensures the quality of the intended learning outcomes, the curriculum, the 
staff, the services and facilities, the assessments and the learning outcomes achieved 
through regular evaluations. The programme also collects management information 
regarding the success rates and the staff-student ratio. 

Unsatisfactory, satisfactory (weighted and substantiated). 

Programme committees, examining boards, staff, students, alumni and the relevant 
professional field of the programme are actively involved in the programme's internal quality 
assurance. 

Unsatisfactory, satisfactory (weighted and substantiated). 



Assessment 

777e programme has an adequate assessment system in place. 

The level achieved is demonstrated by interim and final tests. The tests and assessments 
are valid, reliable and transparent to the students. 

Unsatisfactory, satisfactory (weighted and substantiated). 



Graduation guarantee and financial provisions 

Tfte institution guarantees students that they can complete the entire curriculum and makes 
sufficiënt financial provisions available. 

The graduation guarantee spans a reasonable period of time that is related to the length of 
the studies. 

Unsatisfactory, satisfactory (weighted and substantiated). 



General conclusion 

The quality of the programme is 

Unsatisfactory, satisfactory (weighted and substantiated). 
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The assessment is based on the following definitions. These definitions relate to both the 
scores obtained for the individual standards and the overall scores awarded to the 
programme. 

Generic quality 

The quality that can reasonably be expected in an international perspective trom a higher 
education bachelor's or master's programme. 

Unsatisfactory 

The programme does not satisfy the generic quality standards. 
Satisfactory 

The programme satisfies the generic quality standards. 

For programmes that are not entirely new and programmes that are being converted, the 
learning outcomes achieved will be factored into the judgement. 



6.3 Composition of the assessment panel 19 

NVAO convenes and appoints the assessment panel that will conduct the initial 
accreditation. The programme to be assessed is entitled to lodge substantiated objections to 
the composition of the assessment panel. 

It is imperative that assessment panels are composed in a manner allowing meaningful 
discussions among peers, in which the panel remains sufficiently independent. 



The panel secretary has completed NVAO training leading to certification. Every year, 
NVAO publishes a list of NVAO certified secretaries. 



Assessment panels must meet the following requirements. 



1 . The panel is composed of a minimum of four members, among whom at least two 
authoritative domain experts 20 and a student. 



2. Overall, the panel commands the following expertise: 

a. expertise regarding developments in the discipline, 

b. international expertise, 

c. practical expertise in the professional field relevant to the programme (if 
applicable), 

d. experience in teaching and educational development at the relevant 
programme level and expertise regarding the teaching format(s) used in 
the programme 21 , 

e. student-related expertise, 

f. assessment or audit expertise. 



This paragraph is explained in detail in the guideline titled Eisen aan de panelsamenste/ling [Requirements for the composition of 
panels]. This guideline contains detailed requirements to be met by panels. It also contains a submission procedure, a form to be filled out 
by^the institution and a code of conduct for panel members. 

Domain expertise is understood to mean specialist expertise, international expertise or professional expertise. 

21 

This includes, for example, distance learning, work-related courses, flexible education, skill-oriented education or education aimed at 
excellent students. 
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3. The panel is independent (its members have not had any ties with the institution 
providing the programme for at least the past five years). 

4. The panel is assisted by an independent, external secretary trained and certified by 
NVAO. The secretary does not sit on the panel. 

The assessment panel is counselled by an NVAO process co-ordinator and supported by a 
secretary. The secretary and the process co-ordinator are also independent of the institution 
in question. The secretary and the process co-ordinator do not sit on the panel. 

Prior to the visit, all panel members and the secretary certify to not maintaining any 
connections or ties with the institution in question, either as a private individual or as a 
researcher / teacher, professional or adviser, which could affect an independent judgement 
of the quality of the programme in either a positive or a negative sense, and to not having 
had such connections or ties with the institution during the past five years. 
In addition to the factual independence, as expressed above in the nature of the relation and 
the number of years, it is essential for any panel member or secretary to feel independent. 
In some cases, an independence of more than five years may not provide sufficiënt 
guarantee for an independent position; a prospective panel member or secretary could still 
experience too strong a relationship with the institution or be involved too closely with an 
institution or programme, for example because of family ties. In such cases, the prospective 
panel member or secretary cannot sit on the panel. Panel membership requires a 
professional attitude. To that end, NVAO has formulated a code of conduct for panel 
members and secretaries. This code of conduct comprises elements pertaining to the 
independence, confidentiality and attitude of the panel members and the secretary during 
the assessment process. 

Panel members and secretaries will sign a declaration of independence and confidentiality 
prior to the assessment process. In this declaration, they attest to having taken note of the 
code of conduct. Following the assessment process, the chair and secretary sign the 
assessment report once all panel members have read and approved the report. The report 
includes a declaration that the assessment has been carried out independently. 

Stakeholders such as panel members, staff or students may report to NVAO any matters 
arising during the assessment process that could affect the independence of the 
assessment. 



6.4 Assessment process 

6.4. 1 Information dossier 

For the purpose of the assessment by the assessment panel, the programme presents an 
information dossier regarding the programme. The information dossier should follow the 
standards outlined for the extensive initial accreditation framework. It positions the 
programme in relation to existing (and new) programmes at home and abroad. The critical 
reflection is a self-contained document that can be read separately. 

The information dossier is the pre-eminent tooi to allow teachers and peers to comment on 
the contents of the programme. It must be a document reflecting the commitment of the 
stakeholders. 
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In addition, the assessment framework offers opportunities to discuss the ambitions of the 
programme during the site visit. What choices will the programme make for the future, what 
direction will it take? In order to conduct such discussions, the assessment panel is 
expected to be able to reflect on the programme's plans for the future, together with the 
programme's representatives. 

The information dossier comprises a maximum of 35 pages, excluding appendices. 

6.4.2 Site visit 

In principle, the required site visit for the purpose of extensive initial accreditations takes one 
day. Prior to the visit, the panel members will have formed a preliminary opinion about the 
programme and drawn up questions for their site visit. 

During the site visit, the assessment panel meets with the (prospective) programme 
management, the (prospective) members of the examining board and the programme 
committee, (prospective) teachers and, wherever relevant, representatives of the 
professional field. In addition, the panel examines the material made available by the 
programme. The panel determines the exact scope of the discussions, the possible 
clustering of discussion participants and the further organisation of the visit. The panel 
decides at its own discretion which teachers and students it would like to see and which 
documents it would like to examine. In principle, the programme delegations comprise no 
more than six persons. 

At the end of the site visit, the chair of the assessment panel provides brief feedback 
information to the programme regarding the general judgement and the underlying 
considerations. 

6.4.3 Assessment procedure within the assessment panel 

The assessment panel presents its judgement regarding all the standards incorporated in 
the assessment framework. This judgement is substantiated by an appraisal of the positive 
and critical elements from the panel's findings. The judgement may be: unsatisfactory or 
satisfactory. The panel subsequently formulates a general, weighted and substantiated 
judgement regarding the quality of the programme. This judgement is given on the same 
two-point scale (unsatisfactory or satisfactory). 

6.4.4 Advisory report 

The assessment panel secretary draws up an advisory report comprising 20 to 30 pages. 
The main content of the report is made up of the panel's judgements regarding the 
standards, including underpinnings based on the programme's information dossier, the 
meetings with representatives of the programme and the underlying data from the 
documents studied. The report will include significant and representative examples. In the 
report, the panel gives an account of the manner in which it has organised its visit and how 
it has arrived at its choice of discussion partners and documents. 

The assessment report is preceded by a summary judgement regarding the quality of the 
programme comprising a maximum of two pages. Any measures for improvement will be 
presented in a separate paragraph. In addition, the report contains a score table with the 
panel judgements, information on the date(s) of the site visit, the names of the discussion 
partners, basic data concerning the programme (see paragraph 6.6), an overview of the 
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material studied and the declarations of independence signed by the panel members and 
the secretary. 

NVAO forwards the advisory report to the board of the institution once all panel members 
have approved its contents. The institution is given a term of two weeks to respond to any 
factual inaccuracies in the report, whereupon the panel chair endorses the report after all 
panel members have taken note of and approved its contents. The report is signed by the 
chair and the secretary of the panel and submitted to NVAO for decision-making. If NVAO 
finds that a report raises questions or if an institution so desires, NVAO may invite the 
programme and/or the assessment panel for further consultations. 



6.5 NVAO decision-making 

Basically, NVAO can take two decisions: a positive initial accreditation decision for a period 
of six years, or a negative initial accreditation decision. 

In special cases, NVAO may attach conditions to its decision. In that case, the programme 
must apply for additional assessment within a year, whereupon NVAO ascertains whether 
the programme meanwhile meets the conditions set. If the programme fails to apply for an 
additional assessment or does not meet the conditions, the positive decision expires. 
Satisfaction of the conditions set will be assessed by an assessment panel commissioned 
by NVAO. The additional assessment will basically be carried out in accordance with the 
procedure for regular limited initial accreditations. The assessment panel will focus on the 
programme's shortcomings identified earlier. 



6.6 Required documents 

During the assessment process, the programme provides the assessment panel with a 
limited number of documents. NVAO assumes that these are existing documents, available 
within the programme or the institution, rather than documents prepared especially for the 
programme assessment. The documents serve as a substantiation and if need be as 
verification. Other material is only required when explicitly requested by the panel or if the 
programme wishes to demonstrate a particular distinctive feature. 

6.6. 1 Basic data concerning the programme 

(The basic data is incorporated into the information dossier, the advisory report and the 
NVAO decision.) 

Administrative data regarding the programme 

1. Nomenclature of the programme; 

2. Orientation and level of the programme; 

3. Number of credits; 

4. Specialisations; 

5. Location(s); 

6. Mode (s) of study; 
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Administrative data regarding the institution 

1 . Name of the institution; 

2. Status of the institution (publicly funded or legal body providing higher education). 

Quantitative data regarding the programme 

1 . Intended teacher-student ratio; 

2. Intended amount of face-to-face instruction per stage of the study programme (a stage 
can be expressed in, for example, regular years of study, the work placement and the 
graduation period). 

6.6.2 Required appendices to the information dossier 

(The list of appendices studied will be incorporated into the advisory report.) 

1 . Subject-specific reference framework and the learning outcomes of the programme; 

2. Overview of the curriculum in diagram form; 

3. Outline description of the curriculum components for the first year, stating learning 
outcomes, attainment targets, teaching method(s), assessment method, literature 
(mandatory/recommended), teacher and credits; 

4. Teaching and examination regulations; 

5. Overview of allocated staff with names, positions, scope of appointment, level and 
expertise; 

6. If so required, the macro-efficiency decision; 

7. Overview of the contacts maintained with the professional field (if relevant). 

6.6.3 Documents to be made available during the visit 

(The list of documents studied will be incorporated into the advisory report.) 

1 . Education policy plan or similar document(s); 

2. Policy plan regarding research in relation to the programmes offered or similar 
document(s); 

3. Staff (policy) plan or similar document(s); 

4. Services and facilities plan or similar document(s); 

5. Policy plan regarding the accessibility and feasibility for students with a functional 
disability; 

6. Quality assurance plan; 

7. Reports on consultations in relevant committees / bodies; 

8. Reference books and other learning materials. 
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7 Distinctive features 



7.1 Background 

The distinctive features have been incorporated into the accreditation system because they 
can contribute to the national and international profiling of higher education programmes. A 
distinctive feature enables institutions to draw attention to aspects that are not directly 
related to programme levels but involve, for example, the orientation of a programme (such 
as research master's programmes), objectives such as sustainability or the residential 
nature. 

Distinctive features are assessed on the basis of the following principles: 

1 . The audit panel or assessment panel assesses a distinctive feature by reference to the 
relevant framework in combination with the criteria set out below. The panel ascertains 
whether the institution or programme profiling the distinctive feature fulfils its promise. 

2. To that end, the panel determines, in consultation with the institution or programme, 
what standards it will focus on during the assessment. 

3. The required comparison with other relevant institutions or programmes is performed 
by the institution or programme itself. 

4. The composition of the assessing panel is geared to the assessment of the distinctive 
feature. 

5. The point of departure is that an institution or programme may apply for assessment of 
a distinctive feature at any time. However, its accreditation period may not exceed the 
final date of the original application. 

6. A distinctive feature must meet the following criteria. 
7. 

7.2 Criteria for distinctive features 



Distinguishing nature 

cnterion 1: The distinctive feature distinguishes the institution or programme trom other relevant 
institutions or programmes in the Dutch higher education sector. 

Expianation: The institution or programme demonstrates that the distinctive feature has a distinguishing 
but not necessarily unique nature vis-a-vis relevant institutions or programmes in the Dutch 
higher education sector. 

Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 
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Concretisation 

cnterion 2: The impact of the distinctive feature on the quality of the education provided has been 
operationalised on the basis of the relevant standards in the appropriate assessment 
framework. 

Expianation: The assessing panel indicates which standard(s) it regards as relevant to the realisation of 
the feature and why. The judgement must demonstrate the operationalisation of the 
distinctive feature for the relevant standard(s). If a distinctive feature spans several 
standards in the framework in question, the judgement should provide a concrete and 
complete assessment of the feature for all standards concerned. 

Judgement: Meets, does not meet or partially meets the Standard (weighted and substantiated). 



General conclusion 

The distinctive feature is 
Judgement: Granted, not granted (weighted and substantiated). 
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8 Assessment scales for programme assessments 



Each judgement is illustrated with a number of examples to assist in its operationalisation. 
Examples that apply exclusively to extensive programme assessments are marked "EPA". 

8.1 Unsatisfactory 

The programme does not meet the current generic quality standards and shows serious 
shortcomings in several areas. 

This judgement could be operationalised as follows: 

• The level and/or orientation of the learning outcomes do not fit within the (inter)national 
qualification frameworks and have not been concretised into subject- or programme- 
specific performance levels. 

• The aggregate of curriculum, staff, services and facilities does not constitute an 
environment conducive to learning. 

• The programme lacks a programme-wide, transparent and coherent assessment 
policy. 

• The intended learning outcomes are not being achieved. 

• Quality assurance in the programme is not pursued in a systematic manner, which 
translates into a lack of improvement policy (EPA). 

8.2 Satisfactory 

The programme meets the current generic quality standards and demonstrates an 
acceptable level across its entire spectrum. 

This judgement could be operationalised as follows: 

• The level and/or orientation of the learning outcomes fit within the (inter)national 
qualification frameworks and have been concretised into subject- or programme- 
specific performance levels. 

• The aggregate of curriculum, staff, services and facilities constitutes an environment 
conducive to learning which enables students to achieve the learning outcomes. 

• The programme has developed a programme-wide, transparent and coherent 
assessment policy, which, however, is not yet pursued by all parties involved. 

• The intended learning outcomes are achieved. 

• Quality assurance in the programme is pursued in a systematic manner, which 
translates into a consistent improvement policy (EPA). 
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8.3 Good 

The programme systematically surpasses the current generic quality Standard across its 
entire spectrum. 

This judgement could be operationalised as follows: 

• The level and/or orientation of the learning outcomes fit within the (inter)national 
qualification frameworks and have been concretised into subject- or programme- 
specific performance levels. These are given a specific interpretation based on the 
programme's explicit views. 

• The aggregate of curriculum, staff, services and facilities constitutes a challenging 
learning environment. 

• The programme has developed a programme-wide, transparent and coherent 
assessment policy, which is pursued by all parties involved. 

• The learning outcomes achieved translate into products that are systematically above 
average. 

• Quality assurance in the programme is pursued in a systematic manner, which 
translates into a consistent improvement policy that is reflected in a growing quality 
culture (EPA). 



8.4 Excellent 

The programme systematically well surpasses the current generic quality standards across 
its entire spectrum and is regarded as an (inter)national example. 

This judgement could be operationalised as follows: 

• The level and/or orientation of the learning outcomes fit within the (inter)national 
qualification frameworks and have been concretised into subject- or programme- 
specific performance levels. These are given a specific interpretation based on the 
programme's explicit and unique views. The programme serves as an example both 
nationally and internationally. 

• The aggregate of curriculum, staff, services and facilities constitutes an innovative, 
original learning environment. 

• The learning outcomes achieved are of excellent quality and translate into awards and 
(inter)national publications. 

• Quality assurance in the programme is pursued in a systematic manner, which 
translates into a consistent improvement policy and a strong ability for self-reflection. 
This is reflected in a robust quality culture (EPA). 
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9 Assessment rules 



9.1 Programme assessments 

For programmes offering various modes of study (for example, full-time, part-time and work- 
based learning), the assessment must demonstrate that the generic quality of each mode of 
study is assured, based on the standards in the relevant assessment framework, in order to 
arrivé at a positive final conclusion regarding the programme. 

Programmes that are offered at various locations under a single CROHO registration only 
qualify for accreditation if the assessment shows that each location meets the generic 
quality standards stated in the relevant framework. 

Limited programme assessments 

• The final conclusion regarding a programme will always be "unsatisfactory" if Standard 
3 is judged "unsatisfactory". In case of an unsatisfactory score on Standard 1 , NVAO 
cannot grant an improvement period. 

• The final conclusion regarding a programme can only be "good" if at least two 
standards are judged "good"; one of these must be Standard 3. 

• The final conclusion regarding a programme can only be "excellent" if at least two 
standards are judged "excellent"; one of these must be Standard 3. 

Extensive programme assessments 

• The final conclusion regarding a programme will always be "unsatisfactory" if Standard 
1 or Standard 1 6 is judged "unsatisfactory". In case of an unsatisfactory score on 
Standard 1 , NVAO cannot grant an improvement period. 

• The final conclusion regarding a programme can only be "good" if at least standards 1 , 
3, 6, 9,13, 14, 15 and 16 are judged "good". 

• The final conclusion regarding a programme can only be "excellent" if standards 1 , 3, 
6, 9, 13, 14, 15 and 1 6 are judged "excellent". 

Limited initial accreditations 

The final conclusion regarding a programme will always be "unsatisfactory" if standards 1 or 
3 are judged "unsatisfactory". In case of an unsatisfactory score on standards 1 or 3, NVAO 
cannot grant a conditional initial accreditation. 

Extensive initial accreditations 

The final conclusion regarding a programme will always be "unsatisfactory" if standards 1 or 
15 are judged "unsatisfactory". In case of an unsatisfactory score on standards 1 or 15, 
NVAO cannot grant a conditional initial accreditation. 

9.2 Institutional quality assurance assessments 

The final conclusion following institutional quality assurance assessments will always be 
"negative" if standards 1 or 4 are judged "does not meet the Standard". In case of an 
unsatisfactory score on standards 1 or 4, NVAO cannot grant a conditional pass on the 
institutional quality assurance assessment. 
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10 Accreditation Decision under the Higher Education and 
Research Act 

In this chapter, NVAO outlines the rules laid down by implementing regulations [Dutch: 
AMVB] regarding conditional decisions and the granting of improvement periods. The Dutch 
Higher Education and Research Act [WHW] stipulates that implementing regulations be 
formulated to specify the conditions under which and the situations in which improvement 
periods may be granted in the accreditation of programmes (Article 5a. 12a, first paragraph), 
the conditional initial accreditation of programmes (Article 5a.11, fourth paragraph) and 
conditional institutional quality assurance assessments (Article 5a.13d, sixth paragraph). In 
this document, this implementing regulation is referred to as: Accreditation Decision under 
the Higher Education and Research Act. 

10.1 Conditional initial accreditation and institutional quality assurance assessment 

NVAO may attach conditions to an initial accreditation or institutional quality assurance 
assessment if, on the basis of the advice submitted by the panel of experts, it arrivés at the 
conclusion that certain quality aspects are unsatisfactory but can reasonably be remedied 
within a timeframe of one year. 

With regard to initial accreditations, this pertains to both extensive and limited assessments 
(Article 5a.10a, second paragraph, and Article 5a.13g, first paragraph). In a conditional 
initial accreditation of institutional quality assurance assessment, the conditions in question 
relate to the efforts expected from the board of the institution to improve the quality aspects 
that are assessed as unsatisfactory as well as the manner in which these efforts must be 
expended, the manner in which and the timeframe within which the board of the institution 
must ultimately report on these efforts to NVAO and the communication by the board of the 
institution to the students and other stakeholders regarding the conditions set. 

The timeframe to be observed for reporting must logically follow the timeframe allowed to 
implement the improvements. A timeframe shorter than one year may be set if, in the 
opinion of NVAO, the improvements may be realised sooner. Communication is important 
because students must be informed to the full when selecting a study programme. This 
information is also relevant to others, such as employers with whom the institution maintains 
a special relationship and who employ many graduates. 

An initial accreditation application must be denied if the standards of 'Intended learning 
outcomes' or Testing' are judged unsatisfactory. An application for an institutional quality 
assurance assessment must be denied if the standards of 'View of the quality of the 
education provided' or 'Improvement policy' are judged unsatisfactory. In those cases, a 
conditional initial accreditation or institutional quality assurance assessment cannot be 
granted. 

10.2lmprovement period for accreditation 

If NVAO assesses an application for the renewal of an existing accreditation or accreditation 
following an initial accreditation decision and determines that the programme does not meet 
all the required quality aspects, it may decide to renew the existing accreditation or initial 
accreditation and grant a so-called 'improvement period'. This pertains to both extensive 
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and limited accreditation assessments (Article 5a.8, second paragraph and Article 5a.13f, 
first paragraph). 

An improvement period may only be granted if, in the opinion of NVAO, the deficiencies may 
reasonably be remedied within a timeframe of no more than two years. 

The assessment report submitted by the assessment panel is essential in this respect. 
However, if the Standard of 'Intended learning outcomes' is judged unsatisfactory, an 
improvement period cannot be granted and the application for accreditation must be denied. 
This is because a programme's ambitions level must be at least up to par. Generic quality is 
not guaranteed in programmes whose intended exit level is sub-standard; thus, they lack a 
critical quality culture and vision, the basis for good-quality higher education of world-class 
standards. In such cases, granting an improvement period is uncalled for. 

NVAO may set conditions when granting an improvement period. In terms of content and 
function, these conditions correspond to the conditions that may be attached to initial 
accreditations and institutional quality assurance assessments. The difference from 
conditional initial accreditations and institutional quality assurance assessments, however, is 
that the board of the institution is required to submit a new application to NVAO no later than 
six months before the end of the improvement period, viz. an application for a decision to 
determine whether the programme meets the accreditation framework as yet (Article 
5a. 12a, fourth and fifth paragraphs of the Act). 

Similar to the points for improvement in initial accreditations, an improvement period may be 
shorter than two years if, in the opinion of NVAO, improvement may be achieved within a 
shorter space of time. The above timeframe for submitting applications is, however, based 
on the assumption that extensions of the validity by a period of less than one year would be 
improbable. In addition, an unsatisfactory score on the Testing and learning outcomes 
achieved' Standard warrants a maximum improvement period of one year, i.e., any 
improvements on this Standard must be feasible within a year, otherwise renewal of the 
existing accreditation or initial accreditation cannot be granted. The assessment panel 
reviews the manner in which the institution has remedied the deficiencies identified by 
NVAO and determines whether the programme meanwhile scores satisfactorily on all the 
statutory quality aspects. 

By analogy with Article 5a. 2, second paragraph of the Act, the assessment panel that 
reviews the improvement must be approved by NVAO, as does the assessment panel that 
originally assessed the programme. The assessment panel that reviews the improvement 
comprises, as a minimum, two domain experts from the panel that originally assessed the 
programme. 
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11 Appeals 

Before making a decision regarding an institutional quality assurance assessment, limited 
programme assessment, extensive programme assessment, limited initial accreditation or 
extensive initial accreditation, NVAO allows the board of the institution a term of two weeks 
to present its views concerning the intended decision. These two weeks fall within the 
statutory time frame of six months (for institutional quality assurance assessments, limited 
initial accreditations and extensive initial accreditations) orthree months (for limited 
programme assessments and extensive programme assessments) within which NVAO is 
required to make its decision. 

Once ratified, the decision is immediately forwarded to the board of the institution. At the 
same time, NVAO publishes its decision by placing it on its web site. 

NVAO decisions are open to appeal. 

Stakeholders may lodge an internal appeal with NVAO. The time frame for lodging internal 
appeals is six weeks. The processing of the appeal involves a hearing. NVAO makes its 
decision within twelve weeks after receiving the appeal. A decision after appeal may be 
postponed for no more than six weeks. Such postponement is communicated in writing. 
NVAO decisions after appeal are open to external appeals with the Administrative 
Jurisdiction Department of the Council of State. The time frame for lodging external appeals 
is six weeks. In principle, the Department gives its verdict six weeks after the session. This 
term can be extended by a maximum of six weeks. 

Pending the internal or external appeal procedure, the Chair of the Administrative 
Jurisdiction Department of the Council of State may be requested to make provisional 
arrangements if urgency, due to the interests involved, so requires. 
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1 The structure of the accreditation framework 

Under the Accreditation Treaty, NVAO is charged with the (initial) accreditation of existing 
bachelor's and master's programmes in the Netherlands and the Flemish Community. The 
present accreditation frameworkl refers to applications tor accreditation submitted by the 
boards of higher education institutions in the Flemish Community. 

In the accreditation process, NVAO has two methods to assess whether the programme 
sufficiently meets the generic quality standards (clause 58 of the Flemish Higher Education 
Act): 

1 . NVAO either grants accreditation based on a published external assessment report; 

2. or NVAO grants accreditation based on an "accreditation abroad" already obtained by the 
programme, if - at the discretion of NVAO - this accreditation abroad was granted using a 
comparable methodological approach. 

Chapter 2 describes the framework for the assessment of programmes in higher education 
based on an external assessment report. 

To assess whether a higher education programme can be granted accreditation, the 
programme is assessed according to six themes (see chapter 2.1). These themes are: 

1 . Aims and objectives 

2. Curriculum 

3. Staff 

4. Services 

5. Internal quality assurance 

6. Results 

These themes are assessed according to standards and corresponding criteria, with a 
differentiation as to level (bachelor's or master's) and orientation (professional or academie) of 
the programme. 

A higher education institution may also request a quality assessment agency to carry out an 
assessment of distinctive quality features of a programme. Such features can then be included 
in the assessment report. The assessment of distinctive quality features does not affect the 
outcome of an accreditation procedure (see Chapter 2.2). 

To assess a programme, an assessment scale is used and assessment rules are applied that 
are based on a judgement of the underlying standards (see Chapter 2.3). 

The final accreditation decision depends on the Accreditation Organisation's validation of the 
external assessment. To this end, criteria have been adopted for the evaluation of the 
procedure and the assessment report drawn up by the quality assessment agency (see Chapter 
2.4). 2 



Chapter 3 describes the criteria according to which NVAO can declare an international 
accreditation equivalent. These criteria relate to the equivalence of the methodological 



1 Clause 10.1 of the Accreditation Treaty gives NVAO the authority to develop an "assessment framework". Based on this 
framework, NVAO assesses whether a programme sufficiently meets the generic quality standards. Assessment 
frameworks must be ratified by the Flemish government. 

2 Information on the procedure followed with respect to accreditation applications can be found in the Guidelines for 
Accreditations in the Flemish Community, drawn up by NVAO. 
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approach used by the accreditation organisation abroad. If equivalence is established, NVAO 
can accredit the programme involved on that basis. 

Chapter 4 describes the transitional arrangement (period 2005 - 2013) tor the assessment and 
accreditation of converted academie programmes provided by university colleges, concerning 
the inter-relatedness between education and research. 

The explanatory part outlines: 

- the clauses of the accreditation framework; 

- accreditations granted by virtue of law during the transitional period following the introduction 
of the accreditation system. 

Appendix 1 outlines the rules to be followed during the transitional period regarding the criteria 
in the accreditation framework that are relevant to the embedment in research of academie 
bachelor's and master's programmes ensuing from the conversion of two-cycle undergraduate 
programmes at university colleges. 
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2 Accreditation of programmes based on a published external 
assessment 

2.1 Assessment framework 



2.1.1 Aims and objectives 

Standards Criteria 



Level and The aims and objectives of the programme are focused on having the students 

orientation: master: 

bachelor's - general competences such as the capacity for logical thought and reasoning, 

programmes the ability to acquire and process information, the ability for critical reflection 

(professional and project-based work, creativity, the ability to perform simple supervision 

orientation) tasks, the ability to communicate information, ideas, problems and solutions to 

both specialists as well as laymen and a positive attitude towards life-long 

learning; 

- general professional competences such as the ability to work together as part 
of a team, a solution-oriented attitude in the sense of being able to define and 
analyse independently complex problematic situations in professional practice, 
and the ability to develop and apply effective strategies to resolve them and to 
develop a sense of social responsibility in connection with the professional 
practice; 

- specific professional competences at the level of a newly-qualified 
professional. 



Level and 

orientation: 

bachelor's 

programmes 

(academie 

orientation) 



The aims and objectives of the programme are focused on having the students 
master: 

- general competences such as the capacity for logical thought and reasoning, 
the ability to acquire and process information, the capacity for critical 
reflection, creativity, being able to perform simple management tasks, the 
ability to communicate information, ideas, problems and solutions both to 
specialists and laymen and a positive attitude towards life-long learning; 

- general academie competences such as a research attitude, knowledge of 
research methodologies and techniques and the ability to apply them 
adequately, the ability to collect relevant data in order to form an opinion about 
social, academie, scientific and ethical issues, an appreciation of the 
uncertainty, ambiguity and limits of knowledge and the ability to initiate 
problem-driven research; 

- an understanding of basic academie, discipline-related knowledge inherent to 
a certain domain of the sciences or the arts, systematic understanding of the 
key elements of a discipline which includes acquiring coherent and detailed 
knowledge that is inspired partly by the most recent developments in the 
discipline and an understanding of the structure of the discipline and its inter- 
relatedness with other disciplines. 
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Level and The aims and objectives of the programme are focused on having the students 

orientation: master s master: 

programmes - general competences at an advanced level such as the ability to reason and 

act in an academie manner, the ability to handle complex problems, the ability 
to reflect on one's own thoughts and work and the ability to convert this 
reflection into the development of more effective solutions, the ability to 
communicate one's own research and solutions to peers and laymen and the 
ability to develop an opinion in an uncertain context; 

- general academie competences at an advanced level such as the ability to 
apply research methods and techniques, the ability to design research, the 
ability to apply paradigms in the domain of the sciences or the arts and the 
ability to indicate the limits of paradigms, the ability to be original and creative 
with a view to continuously expanding knowledge and insight and the ability to 
collaborate in a multi-disciplinary environment; 

- advanced understanding of and insight into scientific, discipline-specific 
knowledge inherent to a certain domain of the sciences or the arts, insight into 
the most recent knowledge in the subject/discipline or parts thereof, the ability 
to follow and interpret the direction in which theory formation is developing, the 
ability to make an original contribution towards the body of knowledge of one or 
several parts of the subject/discipline and display specific competences 
characteristic to the subject/discipline such as designing, researching, 
analysing and diagnosing; 

- the competences needed for either independent research or the independent 
practice of the arts at the level of a newly-qualified researcher or artist or the 
general and specific professional competences needed for independent 
application of academie or artistic knowledge at the level of a newly-qualified 
professional. 



Subject-/discipline- 

specific 

requirements 3 



The aims and objectives of the programme (expressed in the learning outcomes 
of the students) correspond with the requirements set by professional colleagues, 
both nationally and internationally, and the relevant professional field for a 
programme in the domain concerned (subject/discipline and/or professional 
practice or practice of the arts). In the case of regulated professions, the 
requirements correspond with the regulation or legislation concerned. 

The learning outcomes of professional bachelor's programmes have been 
authenticated by the relevant professional field. 

The learning outcomes of academie bachelor and master's programmes stem 
from requirements set by the academie and/or artistic discipline, international 
academie practice and, for programmes to which this applies, the practice in the 
relevant professional field. 



3 The subject-/discipline-specific requirements operationalise the generic quality standards. In terms of hierarchy, they are subordinate to these 
standards. 
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2.1.2 Curriculum 

Standards Criteria 

Requirements for The proposed curriculum corresponds with the following criteria for a professional or 

professional / academie orientation: 

academie 

orientation Professional orientation (bachelor's programmes): 

- Students develop their knowledge through the study of professional literature, the 
study of material derived from professional practice or practice of the arts and 
through interaction with the professional field, practice of the arts and/or (applied) 
research; 

- The curriculum has verifiable links with current developments in the professional 
field or the discipline; 

- The curriculum ensures the development of professional or artistic competences 
and has verifiable links with current professional practice. 

Academie orientation (bachelor's and master's programmes): 

- Students develop their knowledge through the interaction between education and 
research (including research in the arts) within relevant disciplines; 

- The curriculum corresponds with current developments in the relevant academie 
or scientific discipline(s) through verifiable links with current academie or scientific 
theories; 

- The curriculum ensures the development of competences in the field of research 
and/or the development and practice of the arts; 

- Where appropriate, the curriculum has verifiable links with the current relevant 
professional practice. 



The curriculum is an adequate realisation of the intended learning outcomes of the 
programme with regard to the level, orientation and subject-/discipline-specific 
requirements. 

The intended learning outcomes are properly reflected in the educational goals of 
the curriculum or parts thereof. 

The contents of the curriculum enable students to achieve the intended learning 
outcomes. 



Consistency of the Students follow a curriculum that is coherent in terms of content, 
curriculum 



Workload The actual duration of the programme is assessed and corresponds with the 

statutory standards. 

The intended learning outcomes are attainable because factors relating to the 
curriculum that could hamper study progress are eliminated wherever possible. 



Correspondence 
between the aims / 
objectives and the 
curriculum 
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Admission The structure and contents of the curriculum are in line with the qualifications of the 

requirements incoming students 4 : 

Bachelor's programmes: 

• Secondary school leaving certificate, Diploma van het Hoger Onderwijs van het 
korte type met volledig leerplan, Diploma van het Hoger Onderwijs voor Sociale 
Promotie or a diploma or certificate that is recognised as equivalent in accordance 
with a law, decree, European directive or other international agreement; 

• Predefined conditions set by the institution for individuals who do not meet the 
above mentioned requirements. 

Advanced bachelor's programmes: 

• A bachelor's degree, with a qualification or qualifications specified in more detail by 
the management of the institution, possibly supplemented with either an assessment 
concerning the fitness or capacity of the individual or a preparatory programme. 

Master's programmes: 

• A bachelor's degree, with a qualification or qualifications specified in more detail by 
the management of the institution, possibly supplemented with an individualised 
curriculum, a preparatory programme or a bridging programme. 

Advanced master's programme: 

• A master's degree, with a qualification or qualifications specified in more detail by 
the management of the institution, possibly supplemented with either an assessment 
concerning the fitness or capacity of the individual or a preparatory programme. 



Credits The curriculum meets the legal requirements regarding the association of credits: 

- Bachelor's degree: at least 180 credits 

- Advanced bachelor's degree: at least 60 credits 

- Master's degree: at least 60 credits 

- Advanced master's degree: at least 60 credits 



Coherence of The educational concept is in line with the aims and objectives. The study methods 

structure and correspond with this educational concept. 

contents 



Learning By means of evaluations, tests and examinations, the students are assessed in an 

assessment adequate manner which is insightful to them to determine whether they have 

achieved the intended learning outcomes of the programme or parts thereof. 



Master's thesis The master's programme is concluded with the master's thesis whereby the student 
demonstrates the ability for analytic and synthetic reasoning, independent problem 
solving at an academie level or artistic creation. The work reflects the general critical- 
reflective attitude or the research attitude of the student. 

The master's thesis corresponds to at least a fifth of the total number of credits with a 
minimum of 15 and a maximum of 30 credits. 



4 The admission requirements are laid down in the Flemish Higher Education Act. With effect from 2005/06, the 
admission requirements pertaining to higher education have been laid down in the Act on Flexible Pathways in Higher 
Education. In terms of content, these admission requirements are identical to those listed here. 
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2.1.3 Staff commitment 

Standards Criteria 

Requirements for The programme meets the following criteria for the deployment of staff for 
professional / a programme with a professional or an academie orientation. 

academie 

orientation Professional orientation: 

- Teaching is principally provided by staff that link the programme to the 
professional practice or practice of the arts. 



Academie orientation: 

- Teaching is principally provided by researchers who contribute to the 
development of the subject/discipline (including research in the arts). 

- In addition, and where appropriate, sufficiënt staff will be deployed with 
knowledge of and insight in the professional field or practice of the arts 
concerned. 



Quantity of staff Sufficiënt staff are deployed to realise the intended quality of the 
programme. 



Quality of staff The staff deployed are sufficiently qualified to ensure that the aims and 
objectives of the programme, in terms of content, didactics and 
organisation, are achieved. 



2.1.4 Services 

Standards Criteria 

Facilities Housing and facilities are sufficiënt for the realisation of the curriculum. 



Tutoring Tutoring and the provision of information to students are adequate in view 

of study progress. 

Tutoring and information provision are geared to students' needs. 



2.1.5 Internal Quality Assurance system 

Standards Criteria 

Evaluation of The curriculum is periodically evaluated in the light of verifiable objectives and 

results other measures. 



Measures for The outcomes of the evaluation form the basis for verifiable measures for 

improvement improvement that contribute to the achievement of the objectives. 



Involvement of Staff, students, alumni and the relevant professional field are actively involved in 
staff, students, the internal quality assurance system. 
alumni and the 
professional field 
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2.1.6 Results 

Standards Criteria 

Achieved learning The achieved learning outcomes correspond with the aims and objectives 
outcomes regarding level, orientation and subject-/discipline-specific requirements. 



Success rate Target figures regarding success rate have been formulated on the basis 

of a comparison with relevant other programmes. 
The programme's success rate is in line with these target figures. 
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2.2 Distinctive quality features 

The aim of accreditation is to establish whether a programme complies with generic quality 
standards. To this end, the programme is assessed according to the assessment framework 
referred to in Chapter 2. In order to emphasise any distinctive quality features, an institution 
may additionally request the Accreditation Organisation to assess distinctive quality features of 
a programme. This can lead to an annotation in the accreditation report that the programme 
does indeed possess this distinctive quality feature. The assessment of a distinctive quality 
feature has no influence on the accreditation decision of the Accreditation Organisation. 

Distinctive quality features must fulfil the following criteria. 



Standards 


Criteria 


Differentiation and 


The feature contributes in a meaningful way to differentiation and profiling in 


profiling 


higher education. 


Quality 


The feature contributes to the quality of the programme. 


Specification 


The effects of the feature for the quality of the programme itself (enrolment, 




curriculum, teaching and learning process, output, services and staff quality) have 




been operationalised. 
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2.3 Assessment rules 

2.3.1 Assessment of the standards 

The assessment of the programme is performed by an external assessment panel (panel of 
experts) co-ordinated by a quality assessment agency that lays down the assessment protocol. 
The assessment panel judges the standards in the assessment framework according to the 
following assessment scale: 

- excellent 

- good 

- satisfactory 

- unsatisfactory. 

2.3.2 Assessment of the programme 

For a positive final conclusion regarding the programme, each theme must be judged as 
satisfactory. The assessment panels judge the themes in the assessment framework according 
to a two-point scale: satisfactory or unsatisfactory. The assessment of a theme in the 
assessment framework is based on the assessments of the separate standards of that theme. 
The assessment panel has to demonstrate clearly how the assessment of the different 
standards led to its final conclusion concerning that theme. In other words, the panel has to 
clarify how - given the criteria in this accreditation framework and the reference framework 
employed - it has arrived at its assessment of a theme on the basis of the analyses of the 
underlying standards. 

In its final conclusion regarding the quality of the programme, the assessment panel needs to 
indicate how its conclusions are based on facts, its analysis of the evidence and its assessment 
of the programme on the basis of this accreditation framework and the reference framework 
employed. In its final conclusion regarding the programme, the assessment panel indicates 
whether, based on the standards in the assessment framework, sufficiënt generic quality 
standards are in place to warrant a positive final conclusion regarding the programme. 

With respect to programmes comprising various modes of study, as referred to in Article 59 ter 
of the Flemish Higher Education Act, a final conclusion can only be positive if the assessment 
shows that sufficiënt generic quality standards are in place for each mode of study. 
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2.4 Assessment rules for accreditation 

2.4.1 Evaluation of external assessments 

The quality assessment agency publishes the report regarding the quality of a programme as 
adopted by the assessment panel. The report also outlines the procedure followed in the 
external assessment. 

The Accreditation Organisation verifies whether the external assessment has been conducted 
in a regular and conclusive manner. 

A regular external assessment publication is structured in accordance with the format outlined 
in the quality assurance protocol. It comprises an assessment based on the generic quality 
standards and the assessment criteria listed in this accreditation framework. 

An assessment report is conclusive if it provides insight into the quality of the composition of the 
assessment panel and permits a thorough investigation into the presence of sufficiënt generic 
quality standards. 

In this manner, a number of procedural and substantive requirements are verified. 

With regard to the procedural requirements, the Accreditation Organisation verifies whether: 

- 1 . the assessment panel followed the assessment protocol laid down by the quality 
assessment agency; 

- 2. the reference framework used by the assessment panel contains the themes and 
standards of the accreditation framework for existing programmes formulated by the 
Accreditation Organisation, including a subject-/discipline-specific interpretation; 

- 3. the assessment panel adequately applied the assessment rules described in this 
accreditation framework; 

- 4. the procedure followed conforms to the assessment framework for the procedures used by 
the VLIR [Flemish Interuniversity Council] and the VLHORA [Council of Flemish University 
Colleges] as quality assessment agencies or in the regulations for the recognition of other 
quality assessment agencies. 

With regard to the substantive requirements, the Accreditation Organisation verifies whether: 

- 1 . the quality judgement of the assessment panel is, in part, based on a comparison with 
similar programmes and, if possible, with international standards applicable to programmes in 
the domain concerned; 

- 2. the report adopted by the assessment panel and published by the quality assessment 
agency contains sufficiënt evidence to verify whether or not the programme fulfils the standards 
of the assessment framework (Chapter 2) and whether sufficiënt generic quality standards are 
in place. The report includes at least the six themes listed in this accreditation framework, 
paying attention to all the standards in each theme. Each Standard is assessed on a four-point 
scale; the themes are assessed based on the underlying assessments of the standards. The 
assessments are substantiated with evidence and analyses as thoroughly as possible. The 
report is finalised with a summarised conclusion on the programme; 

- 3. the report also provides insight into the quality of the assessment panel (the panel of 
experts) that assessed the programme. Information is provided about the size and composition 
of the assessment panel with reference to the knowledge and experience contributed by each 
member, as well as his or her independence, expertise and authority. 
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2.4.2 Assessment rules for accreditation 

If the Accreditation Organisation determines that the external assessment panel took great care 
in formulating its judgement 5 , it addresses the question whether the programme has sufficiënt 
generic quality standards in place. 

In making this decision, the Accreditation Organisation applies the same assessment rules as 
the assessment agency. 

The essence of these assessment rules has been laid down in the Regulation for Decision- 
Making Procedures concerning (Initial) Accreditation of Programmes in the Flemish Community 
[In Dutch: "Reglement tot bepaling van bestuursbeginselen die van toepassing zijn bij de 
besluitvorming inzake accreditatie en toets nieuwe opleiding ten aanzien van opleidingen in de 
Vlaamse Gemeenschap"]. Clauses 7 and 9 of this regulation read: 

"Clause 7. An (initial) accreditation decision is positive if the overall mark for each theme 
described in the (initial) accreditation framework is satisfactory. 

If, at the discretion of NVAO, standards that are rated "unsatisfactory" are balanced out by 
strengths demonstrated within other standards of that same theme, the theme will be rated as 
"satisfactory". 



[...] 



Clause 9. If a programme comprises various modes of study, as referred to in Article 59 ter of 
the Flemish Higher Education Act, a theme can only be rated "satisfactory" if such rating is 
awarded regarding that theme for each separate mode of study. " 



If need be, following depletion of the options for remediation referred to in clauses 58bis and 59bis of the Flemish 
Higher Education Act. 
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3 Accreditation of programmes based on accreditation abroad 

This chapter operationalises the implementation of Clause 60 sexies of the Flemish Higher 
Education Act. This clause stipulates that in the assessment of accreditation applications for 
programmes that have already been granted an international accreditation, the Accreditation 
Organisation must verify whether such international accreditation has been granted in 
accordance with a methodological approach that compares with the accreditations granted on 
the basis of an external review. 

Criteria for equivalence 

General 

The equivalence of a concrete accreditation granted by a foreign accreditation body is 
assessed by criteria for equivalence whose contents can be clustered around five themes: 

- - the foreign accreditation decision is based on a positive assessment of the quality of the 
programme concerned; 

- - the foreign accreditation decision is of a recent enough date; 

- the foreign accreditation decision is based on a public external review; 

- the foreign accreditation organisation has an adequate organisational structure; 

- - the foreign accreditation organisation applies valid quality standards. 

In principle, all criteria for equivalence per theme should be met before the Accreditation 
Organisation can grant an accreditation. 

Theme 1 

The foreign accreditation decision is based on a positive assessment of the quality of the 
programme concerned. 

The foreign accreditation decision or the preceding review should demonstrate that the quality 
of the programme concerned was positively assessed. The international accreditation may 
pertain to other programmes as well, as long as there is a separate positive assessment that 
refers specifically to the programme concerned. 

Theme 2 

The foreign accreditation decision is of a recent enough date. 

The foreign accreditation decision must be of a recent enough date to allow a reasonable 
judgement to be made as to the equivalence. For this reason, the period of time between the 
date on which the accreditation to be recognised as equivalent was granted and the date on 
which the accreditation application was submitted to NVAO must not exceed more than 90 
calendar days. 

Theme 3 

The foreign accreditation decision is based on a public external review 
The foreign accreditation decision must be based on a public external review of the programme 
concerned, of the discipline under which the programme resorts or of the institution offering the 
programme. The external review should be laid down in a public report. 

The external review must be carried out according to previously communicated quality 
guidelines. These guidelines must have been issued by an organisation that functions 
independently from higher education institutions, such as an international association or 
organisation, the legislative or executive powers of the country or state concerned, the foreign 
accreditation organisation itself, an assessment agency recognised by the said accreditation 
organisation or an assessment agency that has signed an administrative agreement with the 
said accreditation organisation in which its independent functioning is stipulated. 
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The external review must be carried out by an independent panel of experts consisting of peers 
and one or more students, unless no candidacy for a student expert was fulfilled and/or student 
involvement in external reviews concerning the programme, the discipline or the institution is 
organised in a different way. 

Among the experts in the panel, at least the following expertise must have been represented: 

- subject-/discipline-specific expertise as to the subject/discipline concerned and the 
professional practice of that discipline. Insofar as relevant, this expertise must be supplemented 
by expertise regarding the international development of that discipline; 

- educational expertise and expertise in the field of didactics; 

- assessment expertise. 

Theme 4 

The foreign accreditation organisation has an adequately functioning organisational structure. 
The international accreditation must have been granted by an autonomous accreditation 
organisation. In this context, "autonomous" means that the political authority of the country or 
state concerned and the institution concerned cannot decisively influence the decision-making 
process of the accreditation organisation. The fact that an appeal can be lodged against 
decisions taken by the accreditation organisation does not, as such, imply that the accreditation 
organisation does not function autonomously. However, a precondition is that an appeal can 
only lead to adjournment or annulment of the disputed accreditation decision. Lodging an 
appeal must never lead to another authority taking a positive or a negative accreditation 
decision. If that were the case, the final accreditation decision would be taken by another 
(public) body and not by the accreditation organisation, which is a prerequisite for autonomy. 

The international accreditation must have been granted by an organisation that operates a 
functioning and continuous accreditation practice according to a set of distinctive working rules. 
The accreditation must not have been granted on an ad-hoc basis. For that reason, the 
accreditation of programmes must be the primary objective (or one of the primary objectives) of 
the foreign accreditation organisation. 

As a guarantee for its continuity, the foreign accreditation organisation must have been 
recognised by a governmental authority. This recognition may be focused on the academie 
and/or professional orientation. 

The foreign accreditation organisation must keep in close touch with its stakeholders in higher 
education in the country or state concerned, so as to be up to date where developments in 
higher education are concerned. 

Finally, the Accreditation Organisation verifies whether the foreign accreditation organisation 
has an apparently adequate internal quality assurance system in place and operates in 
accordance with such a system. 
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Theme 5 

The foreign accreditation organisation applies valid quality standards. 
The foreign accreditation organisation must have assessed the same themes and standards 
that are assessed in the Flemish accreditation framework. This means that the foreign 
accreditation organisation must be able to demonstrate that it has assessed the following: 

- the coherence and relevance of the intended learning outcomes of the specific programme or 
group of programmes or the manner in which this coherence and relevance is guaranteed for 
the programmes offered at the institution; 

- the coherence of the curriculum of the specific programme or group of programmes or the 
manner in which the curriculum is structured within the institution; 

- the quality of the staff deployed for the specific programme or group of programmes or for the 
programmes as a whole offered at the institution; 

- the quality of the services and facilities for that specific programme or group of programmes 
or for the programmes as a whole offered at the institution; 

- the realisation of an internal quality assurance system as regards the specific programme or 
group of programmes or the programmes as a whole offered at the institution; 

- the quality of the achieved learning outcomes of the specific programme or group of 
programmes or of the programmes as a whole offered at the institution. 

The foreign accreditation organisation must have based its quality assessment on the Dublin 
descriptors regarding the quality of higher education or another coherent set of criteria that 
does not differ substantially from the Dublin descriptors. 

The international accreditation should give a clear indication as to the professional or academie 
orientation of the programme. 

In its assessment, the foreign accreditation organisation should have demonstrably verified the 
extent to which the specific programme or group of programmes or the programmes as a whole 
offered at the institution meet broadly accepted and well-documented subject-/discipline- 
specific requirements. 



Page 17 NVAO I September 2009 



nederlands- viaamse accreditatie organisatie 



4 Transitional arrangement for the assessment and 
accreditation of converted academie programmes at 
university colleges, with regard to the inter-relatedness of 
education and research (2005-201 3 period) 

4.1 Introduction 

An essential characteristic of a bachelor or master's programme with an academie orientation is 
that the programme reflects a sufficiënt degree of: 

- academie support and 

- inter-relatedness between education and research. 

This hallmark of its academie nature, comprising the two aforementioned aspects 6 , is 
hereinafter referred to as "embedding education in research" or "embedment in research". 
Various criteria, standards and themes of the assessment framework contained in the 
accreditation framework present the requirement for programmes with an academie orientation 
of reflecting such characteristics. 

Two-cycle university college programmes that have been converted to programmes with an 
academie orientation will not be sufficiently embedded in research right from the introduction of 
the converted programme. Undoubtedly, conversion to programmes with an academie 
orientation is a difficult task for many of these programmes, especially with regard to the 
aspects mentioned above. These programmes will be embedded in research in the years 
ahead. The term "embedding in research" denotes the process by which the educational 
content of a programme is gradually embedded in research until the programme meets all the 
requirements set for programmes with an academie orientation. This embedding process 
should be finalised by the end of the academie year 2012/13. In accreditation procedures based 
on external assessment reports published after 1 January 2005, the academie nature of the 
programmes under consideration is assessed under a transitional arrangement. In particular, 
the requirements for embedment in research will be lowered for these programmes in the 
period up to 2013. As regards all the other standards and themes in the accreditation 
framework, these programmes must meet all the requirements of the accreditation framework 
with effect from 2005, just like other programmes with an academie orientation. 

This transitional arrangement has been laid down as a supplement to Clause 124 § 9 in the 
Flemish Higher Education Act: 

Clause 124 § 9 The Accreditation Organisation assesses the presence of sufficiënt generic 
quality standards in programmes with an academie orientation provided by university colleges, 
as referred to in § 1 and §4 and in Clause 125 ter, taking into consideration the transitional 
nature of the academie support and the intertwining of education and research in the 
programmes, provided the university college has submitted the accreditation dossier bef ore the 
end of the 2012/13 academie year. 

This chapter aims to clarify the transitional arrangement with regard to the assessment 
elements in the accreditation framework. To that end, we will first explore which themes of the 
accreditation framework comprise standards or criteria that are directly related to a 
programme's embedding in research. Subsequently, we discuss how the criteria involved are 
assessed under the transitional arrangement outlined above. 



6 The first aspect primarily pertains to the academie "environment" of the programme, i.e., mainly the manner in which 
teaching staff are actively involved in research. The second aspect relates to the manner in which students are brought into 
contact with research. 
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This transitional arrangement applies to programmes with an academie orientation provided by 
university colleges and resulting from the conversion of two-cycle programmes, for which no 
assessment reports have been published by the end of 2004. It also applies to programmes 
with an academie orientation provided by university colleges and resulting from the conversion 
of postgraduate programmes or the conversion of those departments of adult education centres 
that have been taken over by university colleges. This transitional arrangement does not apply 
to the programmes with an academie orientation resulting from the conversion of two-cycle 
university college programmes whose assessment report has been published before the end of 
2004. These programmes may be granted transitional accreditation until 1 October 2012 or 1 
October 2013, depending on whether the assessment report was published in 2003 or 2004. 
For these programmes, a progress test will be organised by the end of 2007 to assess the state 
of affairs with regard to their embedding in research. This progress test will be conducted under 
the responsibility of the Recognition Commission. In order to be granted accreditation, these 
programmes must meet all of the requirements in the accreditation framework by 2012 or 2013, 
respectively. 

4.2 Related themes and standards of the accreditation framework 

A study of all the themes, standards and criteria in the accreditation framework shows that the 
following themes are directly related to embedment in research: Aims and objectives, 
Curriculum, Staff commitment, Services. The following standards are relevant in this regard: 

- Aims and objectives 

- Level and orientation, bachelor's programmes (academie orientation) 

- Level and orientation, master's programmes 

- Subject-/discipline-specific requirements 

- Curriculum: 

- Requirements for professional and academie orientation 

- Relationship between aims and objectives / curriculum content 

- Master's thesis 

- Staff commitment 

- Academie orientation 

- Services 

- Facilities (research-related infrastructure relevant to teaching) 

Appendix 1 presents all the themes, criteria and standards that pertain to programmes with an 
academie orientation. The criteria or sub-criteria printed in italics denote those that are relevant 
with regard to the assessment of the transitional nature of the programme's embedding in 
research. 

4.3 Assessment of the transitional nature 

The essence of the transitional assessment is that the requirements for criteria related to 
embedment in research are lowered during the transitional period (2005-2013). A general 
principle in the transitional arrangement is that the programme must make a reasonable case, 
on the basis of its progress at the time of assessment and its plans with regard to the further 
embedding of its curriculum in research, which by 2013 will fully meet all the requirements in 
the accreditation framework. 

The procedure implemented for the accreditation of these programmes in the period from 2005 
to 2013 corresponds to the general procedure for the accreditation of existing programmes, as 
outlined in the draft accreditation framework for existing programmes. Along with the plans for 
the further embedment in research, the conversion dossier (including the additional information 
requested by the Recognition Commission) constitutes a source of information for the 
assessment panels in this respect. These panels are responsible for assessing: 
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- whether the progress made in embedding the programme in research, up until the time of the 
external assessment, corresponds to the intentions specified in the conversion dossier and the 
plans tor further embedment in research; and 

- whether the progress made and the further plans make a reasonable case that the 
programme will fully meet the requirements for embedment in research laid down in the 
accreditation framework by 2013 at the latest. 

However, with effect from October 2013, only the actual progress made will be taken into 
consideration in the accreditation of these programmes. No account will be taken of potential 
progress by way of compensating for any failure in building up sufficiënt capacity. 

The theme of "Aims and objectives" 7 warrants a specific approach in this respect. A conversion 
to programmes with an academie orientation is obviously intended to have the programmes 
meet the criteria the accreditation framework sets for the aims and objectives of programmes 
with an academie orientation. On the other hand, most programmes will not yet be able to fully 
realise this academie nature. For that reason, feasible aims and objectives will be drawn up 
during the transition period, which will gradually evolve into the aims and objectives for 
programmes with an academie orientation. This means that in the assessments, a distinction 
will be made between: 

- long-term aims and objectives: the aims and objectives of the programme (with an academie 
orientation) with regard to competencies to be achieved following completion of the embedment 
process; 

- short-term aims and objectives: the aims and objectives of the programme with regard to 
competencies sought to be achieved during the transition period. When assessing the 
programme, the assessment panels must verify whether the programme is achieving these 
short-term aims and objectives and whether it is likely to achieve its long-term aims and 
objectives by 2013 at the latest. 

In the years ahead, the conversion process will gradually be accomplished. Bachelor's 
programmes with an academie orientation will be introduced in the 2004/05 or 2005/06 
academie years, the subsequent master's programmes will start in the 2007/08 or 2008/09 
academie years. This must be taken into account in the assessment. For the accreditation of 
the university college programmes involved this means in concrete terms: 

- that the initial years will mainly focus on descriptions of the aims and objectives, the 
curriculum and the intentions with regard to staff commitment and services/facilities related to 
embedment in research, 

- that gradually more attention will be focused on the progress made with regard to 
embedment in research, for the bachelor's programmes from the 2005/06 or 2006/07 academie 
years onward and for the master's programmes from the 2008/09 or 2009/1 academie years 
onward. 

4.4 Scope of accreditation criteria in the transition period 

The supplement to the Higher Education Act stipulates that university college bachelor's and 
master's programmes (academie orientation) to which the transitional arrangement applies will 
be granted accreditation for four years when they apply for accreditation under the transitional 
arrangement before the end of the 2008/09 academie year. Such programmes applying for 
accreditation after this date will be granted accreditation for six years. For that reason, a 
distinction is made between the period from 2005 to 2008 and the period from 2008 to 2012 in 
the application of the accreditation criteria. This defines the progress programmes must achieve 
in each period in order to make a reasonable case that they will fully meet the requirements set 
down in the accreditation framework by 2013. This means that these requirements involve 



7 The term "aims and objectives" in the Flemish Higher Education Act and the accreditation framework indicates the 
competences the programme seeks to impart on its students. 
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benchmarks rather than strict criteria. Similarly, the point of time indicated should be regarded 
as a reference date rather than a fixed fact. 

The table in Appendix 1 presents an overview of the transitional scope of the criteria from the 
draft accreditation framework that are relevant to the embedment in research of bachelor's and 
master's programmes (academie orientation) resulting from the conversion of two-cycle 
undergraduate programmes provided by university colleges. In the transition period up to 2013, 
the requirements to be met under these criteria will be lowered for the programmes involved. 
No transitional assessment will be implemented for the other criteria, standards and themes 
that are not related to embedment in research. 

The table is structured as follows. 

- Column 1 describes all the themes, standards and criteria from the draft accreditation 
framework for existing programmes relating to programmes with an academie orientation; the 
elements that pertain to embedment in research are printed in italics. 

- Column 2 describes the related scope of the transitional assessment of the criteria or sub- 
criteria pertaining to embedment in research with a view to accreditation of such programmes, 
when the assessment is conducted prior to 2008. 

- Column 3 describes the same for programmes whose assessment is conducted in 2008 or 
beyond. 

- By way of explanation for quality assessment agencies and the Recognition Commission, 
column 4 indicates what information may be used to determine whether the programme meets 
the criteria with regard to embedment in research. This information pertains to both actual 
achievements and plans. The conversion dossier, which specifies the state of affairs regarding 
the programme's embedment in research at the time the dossier is compiled and the intentions 
with regard to the programme's embedment in research, is used as a source of information in 
this respect, along with the plans for continued embedment in research. Other essential 
information documents are the additional policy plans regarding the programme's embedment 
in research, covering the period up to 2013. 

The conversion dossier also serves as a reference for the progress test that will be conducted 
under the responsibility of the Recognition Commission. The progress test is foreseen for the 
end of 2007, which constitutes additional grounds for opting for 2008 as the pivot point in the 
transitional arrangement. 



Page 21 NVAO I September 2009 



nederlands- vlaamse accreditatie organisatie 



Explanatory notes 

General 

Points of departure 

Accreditation is defined as "the formal recognition of a programme based on a decision by an 
independent organisation, stating that the programme fulfils predefined minimum requirements 
regarding quality and level". Such recognition depends on the satisfactory fulfilment of generic 
quality standards. By granting a formal quality mark, accreditation is the culmination of the 
assessment of the quality of programmes. The mark is granted by an Accreditation 
Organisation after validating an external assessment report drafted by an assessment panel co- 
ordinated by a quality assessment agency. The subject of accreditation is the programme. The 
initiative for the application for accreditation lies with the institution. 

By signing the Bologna Declaration, all European countries concerned decided to implement an 
educational model of two cycles in the pursuit of a European Higher Education Area. In line with 
this, many countries implemented accreditation systems in order to determine the level - 
bachelor's or master's - and quality of their programmes. This also bolsters the international 
mobility of students. 

The Accreditation Organisation for Flanders was established through a treaty between the 
Flemish and the Dutch governments. This treaty defines the tasks, composition and 
competence of the Accreditation Organisation. The treaty also regulates how the accreditation 
framework to be set down by the Accreditation Organisation acquires legal force. 

The Accreditation Organisation decides separately on the accreditation of bachelor's and 
master's programmes. This does not alter the fact that related bachelor's and master's 
programmes can be assessed jointly. However, the assessment panel is obliged to give an 
overall mark - satisfactory or unsatisfactory - for each programme (i.e., separately for 
bachelor's and subsequent master's programmes). 

Accreditation framework 

The Flemish Higher Education Act and the Accreditation Framework for existing programmes 
dated 14 February 2003, formulated by the Accreditation Organisation of the Netherlands 
(NAO) were taken as the point of departure for drafting this accreditation framework. The views 
of the organisations consulted prompted us to, on the one hand, endeavour to achieve the best 
possible agreement between the Dutch framework and the Flemish framework, while on the 
other deviating from the Dutch framework when necessitated by stipulations in the Flemish 
Higher Education Act or differences between Flemish and Dutch higher education. Various 
aspects of these Explanatory Notes explicitly refer to the regulations of the Flemish higher 
education system. Similarly, the themes, standards and criteria are explained in the context of 
the Flemish situation. 

Accreditation of existing programmes is based on an assessment of six themes: 

1 . Aims and objectives 

2. Curriculum 

3. Staff commitment 

4. Services 

5. Internal quality assurance 

6. Results 
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These themes are subdivided into standards and criteria, differentiated according to level 
(bachelor's or master's) and orientation (professional or academie) of the programme. 



The table below shows how the themes and standards from the accreditation framework 
correspond to the generic quality standards laid down in the Flemish Higher Education Act 
(Clause 58). 



Generic quality standards laid down in 
the Flemish Higher Education Act 


Accreditation framework for existing 
programmes in Flanders 


Educational contents 

- Nature and level of the curriculum 

- Consistency of the curriculum 

- Workload 

- Correspondence between intended 
learning outcomes and content 


Aims and objectives 

- Level and orientation: bachelor's degree, 
professional higher education 

- Level and orientation: bachelor's degree, 
academie higher education 

- Level and orientation: master's degree 

- Subject-/discipline-specific requirements 

Curriculum: 

- Requirements for professional / academie 
orientation 

- Correspondence between the aims and 
objectives and the curriculum 

- Consistency of the curriculum 

- Workload 

- Admission requirements 

- Credits 


Educational process 

- Harmony between design and contents 

- Student counselling 

- Insightful evaluation and testing 


Curriculum 

- Harmony between design and contents 

- Learning assessment 

- Master's thesis 

Services 

- tutoring 


Material facilities, quality of staff, 
organisation and internal quality assurance 
Methods for self-evaluation 


Staff commitment 

- Requirements for professional / academie 
orientation 

- Quantity of staff 

- Quality of staff 

Services 

- Facilities 

Internal quality assurance 

- Evaluation of results 

- Measures for improvement 

- Involvement of staff, students, alumni and 
the professional field 




Assessment by the assessment panel of the 
quality of the self-evaluation (including the 
methods used) 
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Wherever possible, the subdivision of the accreditation framework, the choice of themes, 
standards and criteria and the level of abstraction applied have been geared to the Dutch 
accreditation framework. 

SubjecWdiscipline-specific framework of reference 

An assessment of the programme cannot be based solely on general criteria pertaining to aims 
and objectives, curriculum, facilities, staff commitment and internal quality assurance. The 
panel must expressly examine whether the programme fulfils the requirements for this specific 
programme and the field of study concerned as set by professional colleagues at home and 
abroad as well as by the professional practice or practice of the arts. In cases of regulated 
professions, account must also be taken of the regulations or rules concerned. The assessment 
panel must therefore be able to assess both the subject-/discipline-specific quality as well as 
the general quality of the programme. This entails specific requirements with regard to the 
composition of the assessment panel. 

In addition, the assessment panel must have a subject-/discipline-specific reference framework 
in the light of which the programme is assessed. For this purpose, the quality assessment 
agency draws up a procedure in the assessment protocol. For academically-oriented 
programmes, connection can be sought with national and international developments in the 
subject/discipline. The Flemish Higher Education Act stipulates that professionally-oriented 
bachelor's programmes must fulfil "profession-specific competences" and that academie 
programmes must fulfil "specific profession-oriented competences required for independent use 
of knowledge". These competences must enable academie graduates to either conduct 
independent research / practice the arts independently or deploy academie / scientific / artistic 
knowledge independently at the level of a newly-qualified professional. These provisions 
constitute the statutory basis for subject-/discipline-specific reference frameworks. Without such 
frameworks, a quality assessment agency is unable to assess whether a programme fulfils the 
requirements demanded of it by the field of study and/or the relevant professional practice. The 
quality assessment agency may adopt any existing professional profiles and curriculum profiles, 
provided these profiles are supported in the sector and the professional field and tie in with 
relevant developments in the professional field and in the field of study. The quality assessment 
agency will have to indicate the procedure for drafting a subject-/discipline-specific reference 
framework. If need be, reference frameworks drawn up for earlier assessments or reference 
frameworks provided by the programmes may be used, insofar as these are sufficiently current 
and carry sufficiënt support within the subject/discipline and relevant national and international 
professional practice. Ultimately, the assessment panel is responsible for establishing a 
subject-/discipline-specific framework. 
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Embedding programmes with an academie orientation in research 

An essential characteristic of programmes with an academie orientation is that the programme 
is supported by research and that teaching and research are intertwined. After all, teaching in 
academically-oriented programmes should be founded on research. In various places in the 
accreditation framework, requirements have been formulated regarding the link with research. 
These requirements refer to: 

- the aims and objectives of the programmes (in accordance with the provisions in Clause 58 
of the Flemish Higher Education Act); 

- the curriculum (requirements regarding academie orientation, such as interaction between 
research and teaching, tying in with academie developments and current scientific theories, 
guaranteeing competences in the area of research and the master's thesis); 

- the staff (teaching is predominantly undertaken by researchers who contribute to the 
development of their subject/discipline); 

- the material facilities concerning the research-related infrastructure relevant to education. 

These requirements apply to all programmes with an academie orientation. Validation of these 
requirements should be undertaken in the light of the research activities of the staff (including 
research projects, doctoral research and scientific output). The interpretation of these 
requirements will depend on the type of programme (bachelor's, master's or advanced master's 
degree) and the phase in the course. As a rule, embedding the programme in research will be 
more pronounced in the master's programmes than in the bachelor's programmes. Research 
competences will be introduced particularly in master's programmes and will be most 
pronounced in the master's thesis. 

As possible aims and objectives of a master's programme, the Flemish Higher Education Act 
mentions on the one hand imparting the necessary academie competences that will enable the 
graduate to conduct independent research or to practice the arts independently at the level of a 
newly-qualified researcher or artist, and on the other, imparting general and specific profession- 
oriented competences necessary for the graduate to use academie or artistic knowledge 
independently at the level of a newly-qualified professional. 

Depending on the subject-/discipline-specific nature of the programme, the requirements for the 
programmes will be specified in more detail. Thus, for instance, in the case of the master's 
programmes that are specifically aimed at training in research (such as "research master's 
programmes"), the subject-/discipline-specific requirements should run parallel with the 
international standards for such programmes. In addition, in assessing the aims and objectives 
of these programmes, more weight is given to the acquiring of competences necessary for 
conducting independent research at the level of a newly-qualified researcher and stricter 
requirements are set for embedding the programmes in research. This means that stricter 
requirements will be imposed regarding various aspects, including the research capacity of the 
staff and their experience in training newly-qualified researchers, a wide research orientation in 
the curriculum, the academie or scientific interpretation of the master's thesis and the 
connection with current academie or scientific developments. 

External quality assessment and accreditation 

The point of departure for the accreditation process is that it is a continuation of the external 
quality assurance. External quality assurance is particularly aimed at improving quality, while 
accreditation is aimed at assessing whether the requirements for generic quality have been 
fulfilled, as described in Chapter 2 of this accreditation framework. 

VLIR [Flemish Interuniversity Council] and VLHORA [Council of Flemish University Colleges] 
are responsible for co-ordinating the external assessment of the statutory registered institutions 
for higher education. All bachelor's and master's programmes organised by these institutions 
must participate in the assessments co-ordinated by VLIR and VLHORA. VLHORA co-ordinates 
the joint external assessment of professionally-oriented bachelor's programmes. VLIR co- 
ordinates the joint external assessment of programmes that are provided by the universities and 
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by the statutory registered institutions that are neither a university nor a university college. 
Together VLIR and VHLORA co-ordinate the joint external assessment of the academically- 
oriented bachelor's and master's programmes that are provided by university colleges within 
the framework of an association, and of the academie programmes jointly organised by a 
university and university college. 

With effect from the external assessments that are conducted on the basis of self-evaluation 
reports completed before 1 September 2010, VLUHR [Council of Flemish Universities and 
University Colleges] will take over the role of quality assessment agency from VLIR and 
VHLORA. 

The Flemish Higher Education Act prescribes clustered external assessment of similar 
programmes at all universities or university colleges and other statutory registered institutions 
within the framework of the external quality assessment by VLIR/VHLORA/VLUHR. 
Assessment panels carry out the external assessment of the programmes or clusters of 
programmes. The quality assessment agencies publish the result of the assessment of the 
programmes in a report adopted and made public by the assessment panel. 

The following procedure for accreditation is incorporated in the Flemish Higher Education Act. 
The Accreditation Organisation decides to grant accreditation, if based on the report of an 
external assessment it is of the opinion that the programme fulfils the generic quality standards. 
The Accreditation Organisation assesses the report of the quality assessment agency. The 
report of the quality assessment agency contains a summarised conclusion on whether the 
programme fulfils the requirements for generic quality, as described in Chapter 2 of this 
accreditation framework. 

External assessments for the purpose of accreditation can be performed by VLIR, VLHORA 
and at a later date, VLUHR or another quality assessment agency recognised by the 
Accreditation Organisation. Alongside VLIR, VLHORA and VLUHR, only quality assessment 
agencies recognised by the Accreditation Organisation may perform external assessments for 
the purpose of accreditation. 

For external assessments relating to external quality assurance that are performed under the 
responsibility of VLIR, VLHORA or VLUHR and used as external assessments for the purpose 
of accreditation, the report and the procedure followed must fulfil the requirements set by the 
Accreditation Organisation. 

It is of the utmost importance that the implementation of the accreditation system does not lead 
to corrosion of the improvement function of the external quality assurance, which may be 
regarded as an important achievement of the former assessment system. Ensuring the purpose 
of quality improvement is the responsibility of VLIR, VLHORA or, as the case may be, VLUHR, 
as part of their statutory task of co-ordinating external quality assurance. 
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Scope 

Accreditation relates to programmes of statutory registered institutions and programmes of non- 
statutory registered institutions. 

Accreditation is a precondition for registering a programme on the Flemish Higher Education 
Register. Registration on the Higher Education Register is a precondition for a programme to 
award recognised bachelor's and master's degrees. For the period 2003 — 2006, the Flemish 
Higher Education Act, in anticipation of a new funding system, provides university colleges and 
universities with individually established ("frozen") budgets. Under the Flemish Higher 
Education Act, only programmes that are registered on the Higher Education Register will be 
eligible for funding. It is to be expected that in the future funding system, the funding of 
programmes provided by statutory recognised institutions will depend on their having attained 
accreditation. 

Under the Flemish Higher Education Act, other institutions may be registered, provide 
recognised bachelor's and master's programmes and award recognised bachelor's and 
master's degrees. A precondition is that the programmes concerned are accredited and 
registered on the Higher Education Register. Programmes that are provided by these registered 
institutions do not have to comply with all the provisions of the Flemish Higher Education Act, 
for instance, the language regulation. These non-statutory registered institutions are not funded 
by the government. 

Embedding programmes in research 

Within the framework of embedding programmes in research, two-cycle university college 
programmes may be converted to academically-oriented bachelor's and master's programmes 
in association with a university. By the end of the period of accreditation by virtue of law (the 
end of the academie year 2012/13), all academie programmes provided by universities and 
university colleges must fulfil the same accreditation requirements. These requirements are 
described in Chapter 2.1 of this accreditation framework. 

A number of two-cycle university college programmes may not be able to shortly fulfil the 
requirements laid down in this assessment framework regarding the criteria pertaining to 
embedding the programme in research. Moreover, within the associations, embedding these 
programmes in research will still take a number of years. For this reason, during the transitional 
period up to 2013, these programmes may request accreditation under a transitional 
arrangement. This transitional arrangement is described in Chapter 4 and applies to 
programmes whose first assessment report was published after 31 December 2004. These 
programmes must apply for accreditation within one year after the publication of this report, at a 
time when the embedment process is still in progress. The time of accreditation in the transition 
period will strongly affect the extent to which the embedment process has already been 
realised. In the period up to 2013, assessment of the embedding in research of programmes 
with an academie orientation provided by university colleges that are making use of the 
transitional arrangement will be based on the extent to which both the criteria from the 
accreditation framework as well as the intentions for embedment of the programmes have been 
realised. This transitional assessment also applies to programmes with an academie orientation 
provided by university colleges that ensue from the conversion of advanced programmes or the 
conversion of programmes that were taken over from adult education centres. 

In the self-evaluation document, the programme must indicate to what extent embedment in 
research has been realised and what steps it is taking to fully meet the criteria from the 
accreditation framework at the end of the transitional period. This is reviewed by the 
assessment panel. For programmes that are accredited under the transitional arrangement, the 
accreditation term is reduced to four years provided the institution submits its application for 
accreditation prior to the end of the academie year 2008/09 and to six years when the institution 
submits its application for accreditation between 1 October 2009 and the end of the academie 
year 2012/13. Programmes that do not wish to appeal to the transitional assessment of 
academic/scientific support and the intertwining of education and research must state this 
explicitly in their self-evaluation. 
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Higher Education in the Arts 

Flanders has a limited number of one-cycle art programmes that are being converted into 
professionally-oriented bachelor's programmes. In addition, there are two-cycle programmes in 
the field of the audio-visual and visual arts, music and performing arts. These programmes are 
being converted into academically-oriented programmes; they must be provided within the 
framework of an association with a university and must fulfil the requirements that are 
demanded of bachelor's and master's programmes with academie orientation, as formulated in 
Chapter 2 of this accreditation framework. This means, among other things, that the programme 
must be intertwined with research and that a substantial number of the teachers must be active 
in research in the arts. 

Advanced bachelor's and master's programmes 

The admission requirements for advanced bachelor's and master's programmes differ from 
those that are in place for regular bachelor's and master's programmes. Apart from that, these 
programmes must fulfil the same quality requirements as other bachelor's and master's 
programmes. 
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Explanatory notes per chapter 

Chapter 1 : structure of the accreditation framework 

The accreditation framework comprises criteria concerning contents along with criteria for the 
procedure followed and the report by the quality assessment agency. The criteria for assessing 
distinctive quality features of a programme are different in status from the other criteria because 
the conclusion about distinctive quality features does not influence the accreditation. 



Chapter 2: accreditation of programmes based on a published external assessment 
report 

Assessment framework 

The assessment framework contains the criteria that the Accreditation Organisation uses to 
examine whether the programme fulfils the requirements for generic quality. 

- Re 2.1. Aims and objectives of the programme 

The level and orientation of the aims and objectives of the programme are examined in the light 
of the learning outcomes for professionally-oriented bachelor's programmes, academically- 
oriented bachelor's programmes and master's programmes as contained in Clause 58 of the 
Flemish Higher Education Act. 

In addition, the aims and objectives are assessed in the light of subject-/discipline-specific 
requirements that are set by the relevant discipline concerned and/or by the professional 
practice or practice of the arts. For professionally-oriented bachelor's programmes, the learning 
outcomes must be assessed with the relevant professional field in the broadest sense, which 
may include the relevant social sector. 

- Re 2.2. Curriculum 

The programme's intended general and subject-/discipline-specific learning outcomes (the 
competences to be attained) must be specified explicitly in the curriculum. The proposed 
contents and design of the curriculum (including the attainment targets of the programme) must 
ensure that incoming students can achieve the intended learning outcomes within the set time. 
The aims and objectives of the programme are assessed through its contents. Professionally- 
oriented bachelor's programmes must accord to relevant developments in professional practice; 
academically-oriented bachelor's and master's programmes must accord to developments in 
the relevant research disciplines. Essential aspects in the assessment of academically-oriented 
programmes are the embedment of the programme in research and the development of 
competences in the area of research (including research in the arts). Academically-oriented 
master's programmes may aim at imparting competences for independent research or 
independent practice of the arts at the level of a newly-qualified researcher or artist, and/or at 
imparting profession-oriented competences for independent use of knowledge at the level of a 
newly-qualified professional. In the latter case, such programmes will have an additional 
professional orientation. Therefore, the requirements for bachelor's and master's programmes 
with an academie orientation include that where applicable the programme must have 
demonstrable ties with the actual practice of the relevant profession. 

The structure of the curriculum is also subject to assessment. The focus is on the 
correspondence between the aims and objectives and the contents of the curriculum, the 
correspondence between the various components of the curriculum, the workload, the 
alignment with the needs of incoming students, the coherence of structure and contents, 
learning assessments (including test formats and the information provided to students about the 
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procedure and the forms of evaluation) and the organisation of the curriculum. With regard to 
the workload, the correspondence between actual and budgeted study time is also assessed. 
The standardisation applied in this framework must be seen in the light of the current statutory 
provisions. The standards included in the framework correspond to the questions posed in the 
current assessments by VLHORA and VLIR. They focus on assessing whether the curriculum 
contributes to the attainment of the aims and objectives. To this end, formulations are used that 
provide sufficiënt room for current developments concerning the standards. 

The quality of the curriculum must also be apparent from its alignment with the qualifications of 
incoming students. In this respect the statutory and regulatory admission requirements are 
important. In addition to fulfilling the formal admission requirements, the programme will have to 
ensure that its curriculum is effectively geared to the average level of incoming students. In this 
respect, the quality of preparatory / bridging programmes and the admissions policy will also be 
assessed. With regard to curricula that are assessed as a model trajectory for achieving the 
aims and objectives of the programme, application of the Flemish Act on Flexible Pathways in 
Higher Education allows individualised trajectories for students, flexibility in learning 
environments, curricula and educational organisation, and the recognition of prior learning 
(competences and qualifications achieved elsewhere). When the occasion arises, these forms 
of flexibilisation, careers guidance and supervision regarding study progress are also taken into 
account in assessing the themes, standards and criteria involved. 

- Re 2.3. Staff commitment 

Regarding the criteria for the deployment of staff, it is important that the expertise of the staff 
employed in the programme 8 is sufficiently in line with the level (bachelor's or master's) and the 
orientation (academie or professional) of the programme. 

This means that in the case of programmes with a professional orientation, sufficiënt staff must 
be able to draw correspondences with professional practice. In the case of programmes with an 
academie orientation, a significant proportion of the staff deployed for the programme must be 
active in research and in addition, wherever relevant, sufficiënt staff must possess experience, 
knowledge and insight into the profession or artistic practice concerned and/or possess 
international expertise. 

The first Standard refers to the requirements for staff given the orientation of the programme 
(professional or academie). The third Standard (quality of staff) refers to the necessity of the 
presence of a range of expertise depending on the aims and objectives of the programme and 
the educational organisation, and to the corresponding staff development policy. 
The Flemish Higher Education Act does not explicitly mention the Standard of quantity of staff 
under generic quality standards. 9 It is included in the accreditation framework because the 
presence of sufficiënt staff is an obvious precondition for generic quality. 

- Re 2.4. Services 

The services must be sufficiënt to realise the curriculum. Services include facilities, tutoring and 
the provision of information to students. Tutoring must be provided throughout the length of the 
course. The provision of information to students and tutoring must be tailored to the 
requirements of the students. Future students must also be informed adequately about 
expectations regarding their competences. 

The facilities include multimedia libraries, laboratories, equipment, study space or subject- 
specific databases. The nature and level of these facilities differ, depending on the character of 
the programme. During assessment, it is verified whether the facilities correspond with the 
generally prevalent level of quality for such facilities. 



This not only includes staff associated with the institution providing the programme but, as the occasion arises, any 
lersons contributing to the programme regardless of their position. 
Dutch legislation also fails to explicitly mention this Standard. 
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- Ad 2.5. Internal quality assurance 

The institution must demonstrate that quality is structurally and permanently monitored and 
improved. The quality of its quality assurance offers the best guarantee that the programme is 
actively engaged in identifying weaknesses and taking measures to improve them. Thus, quality 
assurance is the best guarantee for maintaining and improving the quality established at 
accreditation, throughout the period for which the programme has been accredited. 

Involvement, particularly of staff and students but also of alumni and the professional field, is a 
fundamental feature of internal quality assurance. This involvement should be evident from the 
method of consultation with all parties, their role in decision-making and how their 
recommendations are incorporated in the programme. 

- Ad 2.6. Results 

The ultimate question is whether the intended aims and objectives have been realised. This is 
determined by comparing the achieved learning outcomes with the intended learning outcomes 
for level, orientation and subject-/discipline-specific features. Comparing the achieved learning 
outcomes with the intended learning outcomes also comprises assessing whether the achieved 
learning outcomes of the graduates of the programme have sufficiënt social relevance, as 
mentioned in Clause 58 of the Flemish Higher Education Act. After all, the social relevance of 
the programme is expressed in the aims and objectives of the programme (in particular in the 
subject-/discipline-specific requirements). The success rates must also be acceptable. The 
stipulation contained in the Act (satisfactory success rates) has been operationalised in the 
Standard for success rate by incorporating in the criteria that target figures must be formulated 
in comparison with relevant other programmes. When formulating the target figures, the 
requirements to be demanded of incoming students can also be taken into account, as well as 
the open access to higher education for most programmes and the programme's own profiling 
on distinctive features. The programme must indicate the success rate achieved in its self- 
evaluation in relation to the target figures quoted and the success rate of other relevant 
programmes. 
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Distinctive quality features 

Besides accreditation, which is aimed at establishing the presence of sufficiënt generic quality 
standards, the Flemish Higher Education Act (Clause 59) also offers the possibility of 
commenting on distinctive features of the programme in the accreditation report. In this 
accreditation framework, the following criteria are included for that purpose: 

- the distinctive feature must make a significant contribution to the differentiation and profiling 
within Flemish higher education, i.e., the distinctive feature must lead to greater diversity in the 
design of programmes; 

- the distinctive feature must contribute to the overall quality of the programme, i.e., the 
programme concerned would not be as good without that specific feature (or without all the 
elements that make up this feature); 

- the effects of the distinctive feature on the quality of education must be made operational, 
i.e., it must be clear what the consequences of the feature are for the aims and objectives, 
contents and design of the curriculum and services. 

Prior to the assessment, the institution should apply for assessment of the distinctive feature by 
the quality assessment agency. The institution has to provide the relevant information about the 
standards of the distinctive feature on the basis of which the quality assessment agency can 
arrivé at a conclusion. 

Examples of distinctive features are a pronounced regional or international orientation, a 
specific bond with research, a distinctive educational concept or an extremely effective 
connection to the profession. 

Assessment rules 

The assessment panel gives a summarised assessment of the quality of the programme with a 
positive or negative overall conclusion concerning the presence of sufficiënt generic quality 
standards. The Accreditation Organisation has formulated assessment rules that describe the 
method by which the assessment panel gradually arrivés at this final conclusion. For a positive 
final conclusion the programme must be rated "satisfactory" on all themes from the 
accreditation framework. The assessment panel must provide insight into its considerations at 
all levels - standards, themes and final conclusions - so that it will be clear how the final 
conclusion was reached and how the different standards were weighed. 

The standards are assessed on the basis of a four-point scale: unsatisfactory, satisfactory, 
good and excellent. The descriptors "unsatisfactory" and "satisfactory" are the primary 
benchmark for the assessment. The rating "good" shows that the quality surpasses the generic 
quality. "Excellent" was included in the assessment scale as the rating that exposés "best 
practices", which can have an exemplary function for other programmes. 

Within a theme, individual standards can be rated as "unsatisfactory". However, at the theme 
level, a programme must always earn the mark "satisfactory". If certain standards are rated 
"unsatisfactory" but the overall theme is rated "satisfactory", the assessment panel has to 
provide insight into its considerations in order to enable the Accreditation Organisation to follow 
and evaluate its decision. 

Some programmes comprise various modes of study, such as specialisations, different 
locations, languages,.... Accreditation, however, takes place at the level of the entire 
programme rather than that of the individual modes of study. When a programme is rated as 
unsatisfactory because of a lack of quality in a particular mode of study, the institution 
concerned may exclude that mode of study from the accreditation application. Excluding a 
mode of study from the application for accreditation implies that it can no longer be offered once 
the programme has been accredited. 
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Assessment rules for accreditation 

In its report, the quality assessment agency accounts for the procedure followed during external 
assessment. The Accreditation Organisation assesses the procedure on the basis of the report 
by the assessment panel and the assessment protocol of the quality assessment agency. 
Independence, expertise and authority of the assessment panel convened by the quality 
assessment agency are the essential factors. The expertise comprises knowledge of one's own 
discipline and the professional field concerned as well as expertise in the field of quality 
assessment and expertise of educational issues relating to higher education programmes. The 
Accreditation Organisation also assesses the procedure as regards the contribution of students, 
both during the assessment (has the panel interviewed students enrolled in the programme?) 
and in the composition of the assessment panel. 

The assessment panel has to substantiate its conclusions on the basis of the reference 
framework used (which also includes the subjectVdiscipline-specific requirements) and its 
analysis of the facts. A comparison of the programme with other programmes, preferably 
international, must be part of the procedure. 

If the Accreditation Organisation concludes that the external assessment has been conducted 
in a regular and conclusive manner, it takes a final decision on whether the programme offers 
sufficiënt quality guarantees. 

In this respect, the Accreditation Organisation may, of course, endorse the external 
assessment. In this case, the external assessment constitutes the grounds for the accreditation 
decision. 

However, the Accreditation Organisation may also partly or completely disagree with the 
external assessment report. In that case, the accreditation decision must be based on the 
assessment rules in place for assessment panels (see Chapter 4). The Accreditation 
Organisation shall then explicitly give grounds for its decision in the accreditation report. 
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Explanatory notes to the system of accreditation by virtue of 
law 

For the purpose of introducing the accreditation system, the possibility of accreditation by virtue 
of law has been laid down by decree. The duration of accreditation by virtue of law depends on 
the publication date of the last assessment report. Programmes whose assessment report was 
published between 1 January 1998 and 31 December 2004 are deemed to have been 
accredited up until the end of the academie year beginning in the eighth calendar year following 
the year in which the assessment report was published. Programmes whose external 
assessment was published after 1 January 2005 are deemed to have been accredited up until 
the end of the academie year beginning in the year following that in which the assessment 
report was published. In any case, all accreditations by virtue of law will expire by the end of the 
201 2/1 3 academie year. 

Postgraduate programmes converted into bachelor's or master's programmes will be accredited 
by virtue of law for a period of four years, with the exception of postgraduate programmes that 
have been converted into initial bachelor's or master's programmes through amalgamation with 
an undergraduate programme. In the latter case, the arrangement for converted initial 
bachelor's or master's programmes applies. The period of four years for accreditation by virtue 
of law also applies to bachelor's and master's programmes ensuing from the conversion of 
adult education centre departments taken over by university colleges. 

The Flemish government may adapt the duration of the accreditation by virtue of law or the 
timeframes for application for converted programmes originating from (components of) several 
original programmes or in order to allow for the joint assessment of programmes. 

For programmes whose assessment report was published after 1 January 2005, the external 
assessment must be based on the criteria laid down in this accreditation framework. 

For the new bachelor's and master's programmes, external assessment for the purpose of 
accreditation will only be used to a limited extent in the years ahead. The assessment reports 
on which the accreditation decisions are based will mainly reflect the external assessment of 
the quality of existing programmes, followed by that of the bachelor's and master's programmes 
under construction and the "old" programmes that are being phased out. The converted 
bachelor's and master's programmes will not be reviewed until the end of the transition period. 
This procedure has been set up in order to avoid having to conduct an excessive number of 
assessments at the same time, i.e., once the bachelor's and master's programmes have been 
fully developed. 
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Appendix 1. 

Interpretation of accreditation criteria in transition period for programmes with an academie 
orientation that ensue from the conversion of university college programmes (pertaining to 
embedment in research) 



Themes, standards, criteria 


Specific assessment of designated 


Specific assessment of designated 


Specific information 




elements until 2008 


elements after 2008 





Accreditation framework for existing higher education programmes in Flanders 
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Theme: 1. Aims and objectives 

Standard: 

1. 1 . Level and orientation: bachelor's programmes, 
professional orientation 

Criteria: 

The aims and objectives of the programme are focused on having 
the students master: 

• general competences such as the capacity tor logical 
thought and reasoning, the ability to acquire and process 
information, the ability for critical reflection and project- 
based work, creativity, the ability to perform simple 
supervision tasks, the ability to communicate information, 
ideas, problems and solutions to both specialists as well as 
laymen, and a positive attitude towards life-long learning; 

• general professional competences such as the ability to 
work together as part of a team, a solution-oriented attitude 
in the sense of being able to define and analyse 
independently complex problematic situations in 
professional practice, and the ability to develop and apply 
effective strategies to resolve them, and to develop a sense 
of social responsibility in connection with the professional 
practice; 

• specific professional competences at the level of a newly- 
qualified professional. 



not applicable in this issue 
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not applicable in this issue 



Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Standard: 

1.2. Level and orientation: bachelor's programmes, academie 
orientation 

Criteria: 

The aims and objectives of the programme are focused on having 
the students master: 

• general competences such as the capacity tor logical 
thought and reasoning, the ability to acquire and process 
information, the capacity for critical reflection, creativity, 
being abie to perform simple management tasks, the ability 
to communicate information, ideas, problems and solutions 
both to specialists and laymen and a positive attitude 
towards life-long learning; 

• general academie competences such as a research attitude, 
knowledge of research methodologies and techniques and 
the ability to apply them adequately, the ability to collect 
relevant data in order to form an opinion about social, 
academie, scientific and ethical issues, an appreciation of 
the uncertainty, ambiguity and limits of knowledge, and the 
ability to initiate problem-driven research; 

• an understanding of basic academie, discipline-related 
knowledge inherent to a certain domain of the sciences or 
the arts, systematic understanding of the key elements of a 
discipline which includes acquiring coherent and detailed 
knowledge that is inspired partly by the most recent 
developments in the discipline, and an understanding of the 
structure of the discipline and its inter-relatedness with other 
disciplines. 


From 2004/05 or 2005/06: 
With regard to academic/scientific support and 
intertwining of education and research, the 
long-term aims and objectives of the 
programme relating to the intended 
competences of graduates correspond to 
general aims and objectives of bachelor's 
programmes with an academie orientation 
outlined in the draft accreditation framework 
for existing programmes, including with 
respect to: 

expertise and application of research 

methodologies and techniques 

research attitude 

formation of opinions 

skills related to the academic/scientific 

discipline 

The short-term aims and objectives must 
evolve in a progressive manner in order to 
make a reasonable case that the long-term 
aims and objectives will be achieved by 2013. 


From 2008/09 

As in the preceding column 


Description of short-term and long- 
term aims and objectives of the 
programme 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Standard: 

1.3. Level and orientation, master's programmes 

Criteria: 

The aims and objectives of the programme are focused on having 
the students master: 

• general competences at an advanced level such as the ability 
to reason and act in an academie manner, the ability to handle 
complex problems, the ability to reflect on one's own thoughts 
and work, and the ability to convert this reflection into the 
development of more effective solutions, the ability to 
communicate one's own research and solutions to peers and 
laymen, and the ability to develop an opinion in an uncertain 
context, 

• general academie competences at an advanced level such as 
the ability to apply research methods and techniques, the 
ability to design research, the ability to apply paradigms in the 
domain of the sciences or the arts and the ability to indicate the 
limits of paradigms, the ability to be original and creative with a 
view to continuously expanding knowledge and insight, and the 
ability to collaborate in a multi-disciplinary environment; 

• advanced understanding of and insight into scientific, 
discipline-specific knowledge inherent to a certain domain of 
the sciences or the arts, insight into the most recent knowledge 
in the subject/discipline orparts thereof, the ability to follow 
and interpret the direction in which theory formation is 
developing, the ability to make an original contribution towards 
the body of knowledge of one or several parts of the 
subject/discipline, and display specific competences 
characteristic to the subject/discipline such as designing, 
researching, analysing and diagnosing; 

• the competences needed for either independent research or 
the independent practice of the arts at the level of a newly- 
qualified researcher or artist, or the general and specific 
professional competences needed for independent application 
of academie or artistic knowledge at the level of a newly- 
qualified professional. 


From 2004/05 or 2005/06: 
The long-term aims and objectives of the 
programme relating to the intended 
competences of graduates have at least been 
outlined and correspond to the general aims 
and objectives for master's programmes in the 
draft accreditation framework for existing 
programmes, including with respect to: 

use of research methodologies and 

techniques 

design of research 

use of paradigms 

formation of opinions 

skills related to the academic/scientific 

discipline 

The short-term aims and objectives must 
evolve in a progressive manner in order to 
make a reasonable case that the long-term 
aims and objectives will be achieved by 2013. 


From 2007/08 or 2008/09: 
The long-term aims and objectives of the 
programme relating to the intended 
competences of graduates have been 
elaborated in full and correspond to the 
general aims and objectives for master's 
programmes in the draft accreditation 
framework for existing programmes. 

The short-term aims and objectives must 
evolve in a progressive manner in order to 
make a reasonable case that the long-term 
aims and objectives will be achieved by 2013. 


Description of short-term and long- 
term aims and objectives of the 
programme 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Standard: 

1.4. Subject-Zdiscipline-specific requirements 

Criteria: 

• The aims and objectives of the programme (expressed in the 
learning outcomes of the students) correspond with the 
requirements set by professional colleagues, both nationally 
and internationally, and the relevant professional field for a 
programme in the domain concerned (subject/discipline and/or 
professional practice or practice of the arts). In the case of 
regulated professions, the requirements correspond with the 
regulation or legislation concerned. 

• The learning outcomes of bachelor and master's programmes 
with an academie orientation stem trom requirements set by 
the academie and/or artistic discipline, international academie 
practice and, for programmes to which this applies, the 
practice in the relevant professional field. 


From 2004/05 or 2005/06 (bachelor's 
programmes, academie orientation) and 
2007/08 or 2008/09 (master's programmes) 
respectively: 

The long-term aims and objectives of the 
programme relating to the embedment in 
research correspond to the requirements set 
by national and international peers. 

The short-term aims and objectives must 
evolve in a progressive manner in order to 
make a reasonable case that the long-term 
aims and objectives will be achieved by 2013 
at the latest. 


As in the preceding column 

The short-term aims and objectives must 
evolve in a progressive manner in order to 
make a reasonable case that the long-term 
aims and objectives will be achieved by 2013 
at the latest. 


Description of short-term and long- 
term aims and objectives of the 
programme 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Theme: 2. Curriculum 

Standard: 

2. 1. Requirements for professional and academie orientation 

Criteria: 

Bachelor's programmes with academie orientation and master's 
programmes: 

• Students develop their knowledge through the interaction 
between education and research (including research in the 
arts) within relevant disciplines; 

• The curriculum corresponds with current developments in the 
relevant academie or scientific discipline(s) through verifiable 
links with current academie or scientific theories; 

• The curriculum ensures the development of competences in 
the field of research and/or the development and practice of 
the arts; 

• Where appropriate, the curriculum has verifiable links with the 
current relevant professional practice. 

Standard: 

2.2. Correspondence between the aims / objectives and the 
curriculum 

Criteria: 

• The curriculum is an adequate realisation of the intended 
learning outcomes of the programme with regard to the level, 
orientation and subjectVdiscipline-specific requirements. 

• The intended learning outcomes are properly reflected in the 
educational goals of the curriculum or parts thereof. 

• The contents of the curriculum enable students to achieve the 
intended learning outcomes. 


From 2004/05 or 2005/06 (bachelor's 
programmes, academie orientation) and no 
later than 2007/08 or 2008/09 (master's 
programmes) respectively: 
The long-term description of the curriculum 
shows sufficiënt: 

a. interaction with research (including 
submitting research themes, feedback 
regarding research issues and results in 
education, involving students in 
research) 

b. connection with current 
academic/scientific theories from the 
disciplines involved 

c. development of skills relating to 
research 

The curriculum must evolve in a progressive 
manner in order to fulfil the requirements by 
2013. 

From 2004/05 or 2005/06 (bachelor's 
programmes, academie orientation) (and no 
later than from 2007/08 or 2008/09 (master's 
programmes, academie orientation)): 
the long-term curriculum described is 
increasingly being realised; 
the short-term realisation of the curriculum 
makes it likely that by 2013 the curriculum will 
fully comply with the requirements laid down 
in the accreditation framework with respect to 
the intertwining with research. 


See up to 2008 
In addition: 

From 2008/09 or 2009/1 (master's 
programmes): 

The curriculum described is increasingly being 
realised - the degree of realisation of the 
curriculum makes it likely that by 201 3 the 
curriculum will fully comply with the 
requirements laid down in the accreditation 
framework with respect to the intertwining with 
research. 

The curriculum must evolve in a progressive 
manner in order to fulfil the requirements by 
2013. 

As above 


Short-term and long-term description 
of the curriculum 

Short-term and long-term description 
of the curriculum 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Standard: 

2.3. Consistency of the curriculum 

Criteria: 

• Students follow a curriculum that is coherent in terms of 
content. 

Standard: 

2.4. Workload 

Criteria: 

• The actual duration of the programme is assessed and 
corresponds with the statutory standards. 

• The intended learning outcomes are attainable because factors 
relating to the curriculum that could hamper study progress are 
eliminated wherever possible. 

Standard: 

2.5. Admission requirements 

Criteria: 

The structure and contents of the curriculum are in line with the 
qualifications of the incoming students: 

Bachelor's programmes: 

• Secondary school leaving certificate, Diploma van het Hoger 
Onderwijs van het korte type met volledig leerplan, Diploma 
van het Hoger Onderwijs voor Sociale Promotie or a diploma 
or certificate that is recognised as equivalent in accordance 
with a law, decree, European directive or other international 
agreement ; 

• Predefined conditions set by the institution for individuals who 
do not meet the above-mentioned requirements. 

Master's programmes: 

• A bachelor's degree, with a qualification or qualifications 
specified in more detail by the management of the institution, 
possibly supplemented with an individualised curriculum, a 
preparatory programme or a bridging programme. 

Advanced master's programme: 

• A master's degree, with a qualification or qualifications 
specified in more detail by the management of the institution, 
possibly supplemented with either an assessment concerning 
the fitness or capacity of the individual, or a preparatory 
programme. 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Standard: 

2.6. Credits 

Criteria: 

The curriculum meets the legal requirements regarding the 
association of credits: 

• Bachelor's degree: at least 1 80 credits 

• Master's degree: at least 60 credits 

• Advanced master's degree: at least 60 credits 

Standard: 

2.7. Coherence ofstructure and contents 

Criteria: 

• The educational concept is in line with the aims and objectives. 

• The study methods correspond with this educational concept. 

Standard: 

2.8. Learning assessment 

Criteria: 

• By means of evaluations, tests and examinations, the students 
are assessed in an adequate manner which is insightful to 
them to determine whether they have achieved the intended 
learning outcomes of the programme or parts thereof. 

Standard: 

2.9. Master's thesis: 

Criteria: 

• The master's programme is concluded with the master's thesis 
whereby the student demonstrates the ability for analytic and 
synthetic reasoning, independent problem solving at an 
academie level or artistic creation. The work reflects the 
general critical-reflective attitude or the research attitude of the 
student. 

• The master's thesis corresponds to at least a fifth of the total 
number of credits with a minimum of 15 and a maximum of 30 
credits. 


The long-term draft of the master's thesis is in 
line with the statutory provisions, including 
with respect to the substantial 
academic/scientific foundation. 

The short-term draft must evolve in a 
progressive manner in order to make a 
reasonable case that the requirements will be 
fulfilled by 2013 at the latest. 


(Depending on the first implementation of the 
master's programmes, from 2008/09 or 
2009/1 0): 

The long-term draft of the master's thesis is in 
line with the statutory provisions, including 
with respect to the substantial 
academic/scientific foundation. 

The short-term draft and its realisation must 
evolve in a progressive manner in order to 
fulfil the requirements by 201 3 at the latest. 


Short-term and long-term description 
of the curriculum 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Theme: 3. Staff commitment 

Standard: 

3.1. Requirements for professional / academie 
orientation 

Criteria: 

The programme meets the following criteria for 
the deployment of staff for a programme with a 
professional or an academie orientation. 

• Teaching is principally provided by researchers 
who contribute to the development of the 
subject/discipline (including research in the 
arts). 

• In addition, and where appropriate, sufficiënt 
staff will be deployed with knowledge of and 
insight in the professional field or practice of 
the arts concerned. 

Standard: 

3.2. Ouantity of staff 

Criteria: 

• Sufficiënt staff are deployed to realise the 
intended quality of the programme. 

Standard: 

3.3. Quality of staff 

Criteria: 

• The staff deployed are sufficiently qualified to 
ensure that the aims and objectives of the 
programme, in terms of content, didactics and 
organisation, are achieved. 


It is likely that by 2013 a significant proportion 
of the teaching will be provided by staff that 
are actively engaged in research and thus 
contribute to the development of their 
discipline, based on the actual achievement 
and the intentions with respect to: 

a. staff policy and recruitment policy 

b. the research tasks and the qualifications 
of the workforce (including the extent to 
which a substantial research task has 
been incorporated into vacancies for new 
staff, increase in number of PhD 
students, increase in number of PhDs, 
increase in research staff vis-a-vis overall 
staff) 

c. deployment of active researchers, if any, 
from partners within and outside the 
Association in education 

d. the scope of the research in which 
programme staff participates 


As in the preceding column. 

The initial achievement is already manifest. 


Staff policy: 

recruitment policy 

measures for boosting the research 
qualifications of staff 

Description of workforce 

research workload of staff (number of FTEs and 
ratio vis-a-vis total workload), with breakdown 
by new staff members and doctoral mandates 
multi-year plans 

Description of research policy (structures, 

organisation, strategy) 

Research projects in which programme staff 
participate (including doctoral mandates) 

Qualifications of staff (in FTEs): 
doctorate 

research experience 
current research activities 

Deployment of academie staff from partners: 
within Association 
outside Association 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Theme: 4. Services 

Standard: 

4.1. Facilities 

Criteria: 

• Housing and facilities are sufficiënt for the realisation of the 
curriculum. 

Standard: 

4.2. Tutoring 

Criteria: 

• Tutoring and the provision of information to students are 
adequate in view of study progress. 

• Tutoring and information provision are geared to students' 
needs. 


Based on the actual achievement and the 
intentions, it is likely that the quality and 
quantity of the research facilities 
(infrastructure, equipment, academic/scientific 
databases, etc.) that are essential to 
guarantee the embedment of education in 
research will be sufficiënt by 2013 at the 
latest. 


As in the preceding column 


Research facilities (achievement and 
planning in institution and through 
collaboration within and outside 
Association), related to embedding 
education in research 

Use of funds earmarked for the 
embedment of programmes in 
research and other financial 
resources for research and 
embedding education in research 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Theme: 5. Internal quality assurance 

Standard: 

5.1. Evaluation of results 

Criteria: 

• The curriculum is periodically evaluated in the light of verifiable 
objectives and other measures. 

Standard: 

5.2. Measures tor improvement 

Criteria: 

• The outcomes of the evaluation form the basis for verifiable 
measures for improvement that contribute to the achievement 
of the objectives. 

Standard: 

5.3. Involvement ofstaff, students, alumni and the 
professional field 

Criteria: 

• Staff, students, alumni and the relevant professional field are 
actively involved in the internal quality assurance system. 
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Themes, standards, criteria 


Specific assessment of designated 
elements until 2008 


Specific assessment of designated 
elements after 2008 


Specific information 


Theme: 6. Results 

Standard: 

6.1. Achieved learning outcomes 

Criteria: 

• The achieved learning outcomes correspond with the aims and 
objectives regarding level, orientation and subject-/discipline- 
specific requirements. 

Standard: 

6.2. Success rate 

Criteria: 

• Target figures regarding success rate have been formulated on 
the basis of a comparison with relevant other programmes. 

• The programme's success rate is in line with these target 
figures. 
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1 Structure of the assessment framework 

The assessment framework for the initial accreditation of higher education programmes 
comprises: 

- a framework for the evaluation of themes, standards and criteria 

- assessment rules 

- a description of the initial accreditation procedure, including the criteria for the evaluation 
of the assessment and the initial accreditation report. 

The decision regarding the accreditation of new programmes is founded on an assessment 
based on six themes.^ 

These themes are: 

- aims and objectives of the programme 

- curriculum 

- staff commitment 

- services and facilities 

- internal quality assurance 

- conditions for continuity. 

The above themes are reviewed on the basis of standards and the corresponding criteria 
(see chapter 2). 

Assessment rules have been laid down for initial accreditations (see chapter 3). 

The Accreditation Organisation bases its judgement regarding the new programme on an 
assessment carried out or commissioned by the accreditation organisation. This 
assessment will result in an initial accreditation report. Criteria have been drawn up 
regarding the quality of the initial accreditation report (see chapter 4 on the Initial 
accreditation procedure) 2 

The method of working and procedure to be followed in initial accreditations have been laid 
down by the accreditation organisation. The organisation has opted for a differentiated 
approach. Although all new programmes have to meet the same criteria, the nature of the 
assessment to be carried out may vary depending on the degree of newness of the 
programme in the institution involved and in the Flemish higher education sector, and 
depending on the extent to which the programme is already offered. 3 



1 Article 58 of the Flemish Higher Education Act uses the concept of 
"generic quality standards" in this respect. The explanatory notes to this 
assessment framework clarify the relationship between the generic 
quality standards from the Higher Education Act and the themes and 
standards in the assessment framework. 

2 The assessment framework briefly touches upon the basics of several 
relevant provisions in the Higher Education Act. Please refer to the 
Higher Education Act for the full wording of these provisions. 

3 The procedure for processing applications is outlined in the Guidelines 
for Initial Accreditations in the Flemish Community, drawn up by NVAO. 
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2 Evaluation framework 



2.1 Aims and objectives of the programme 
Standards Criteria 



Level and 
orientation: 
bachelor's, 
professional higher 
education 



The programme aims and objectives are aimed at leading the students to: 
• master general competencies such as thinking and reasoning skills, 
retrieving and processing information, critical reflection and the ability to 
work in a project context, creativity, the ability to perform simple 
management duties, the ability to communicate information, ideas, 
problems and solutions both to specialists and laymen and an attitude 
conducive to lifelong learning; 



• master general professional competencies such as the ability to work in a 
team, the ability to work in a solution-oriented fashion in the sense of 
autonomously defining and analysing complex problem situations in 
professional practice and being able to develop and apply meaningful 
solution strategies and an awareness of the social responsibility that 
ensues from professional practice; 

• master profession-specific competencies at the level of a newly qualified 
professional. 



Level and 
orientation: 
bachelor's, 
academie education 



The programme aims and objectives are aimed at leading the students to: 
• master general competencies such as thinking and reasoning skills, 
retrieving and processing information, critical reflection, creativity, the 
ability to perform simple management duties, the ability to communicate 
information, ideas, problems and solutions to both specialists and laymen 
and an attitude conducive to lifelong learning; 



• master general academie competencies such as an investigative attitude, 
knowledge of and the ability to use research methods and techniques in an 
adequate fashion, the ability to collect the data relevant to the formation of 
a judgement on social, academie and ethical issues, an appreciation of the 
uncertainty, ambiguity and boundaries of knowledge and the skills required 
for a problem-based initiation of research; 



• gain an understanding of the academie basic knowledge specific to a 
particular domain of science or the arts, a systematic knowledge of the 
core elements of a discipline, including the acquisition of coherent and 
detailed knowledge in part inspired by the latest developments in the 
discipline and an understanding of the structure of the field of study and its 
relationship with other fields of study. 
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Level and The programme aims and objectives are aimed at leading the students to: 

orientation: master's • master general competencies at an advanced level, such as the ability to 

think and act in an academie fashion, the ability to deal with complex 
problems, the ability to reflect on their own thinking and working and 
being able to translate that reflection into the development of more 
adequate solutions, the ability to communicate their own research and 
problem solutions to peers and laymen and the ability to form a 
judgement in an uncertain context; 

• master general academie competencies at an advanced level, such as 
the ability to use research methods and techniques, the ability to design 
research, the ability to use paradigms in the domain of science or the arts 
and being able to identify the boundaries of paradigms, the ability to 
demonstrate originality and creativity with a view to the continuous 
expansion of knowledge and insights and the ability to work in concert in 
a multi-disciplinary environment; 

• an advanced understanding of and insight into the academie knowledge 
specific to a particular domain of science or the arts, insight into the latest 
knowledge of the field of study or components thereof, the ability to follow 
and interpret developments in the formation of theories, the ability to 
make an original contribution to the knowledge relevant to one or more 
components of the field of study and the possession of specific skills 
related to the field of study, such as designing, researching, analysing 
and diagnosing; 

• either master the competencies required to independently conduct 
research or independently practise art at the level of a newly-qualified 
researcher or artist; or master the general and specific professional 
competencies required to independently apply academie or artistic 
knowledge at the level of a newly-qualified professional. 



The aims and objectives of the programme (expressed in learning outcomes) 
correspond to the requirements set for a programme in the relevant domain 
(field of study / discipline and/or professional practice or art practice) by 
(foreign) peers and the relevant professional field. For regulated professions, 
such aims and objectives are in keeping with the relevant rules and 
regulations. 

The learning outcomes of bachelor's programmes with a professional 
orientation have been reviewed by the relevant professional field. 

For bachelor's programmes and master's programmes with an academie 
orientation, the learning outcomes have been derived from the requirements 
set by the academie and/or artistic discipline, the international world of 
science and, when appropriate, actual practice in the relevant professional 
field. 



Domain-specific 
requirements 



2.2 Curriculum 

Standards Criteria 



Requirements The curriculum is geared to the following criteria regarding a professional or 

regarding academie orientation: 
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professional and Bachelor's programmes (professional orientation): 
academie orientation • Students develop knowledge through specialised literature, learning 

materials derived from the professional or arts practice and interaction 
with professional practice, arts practice and/or (applied) research; 

• The curriculum has demonstrable links with current developments in the 
field of study / discipline; 

• The curriculum safeguards the development of professional or artistic 
skills and has demonstrable links with current professional practice; 

Bachelor's programmes and master's programmes (academie orientation): 

• Students develop knowledge through the interaction between education 
and research (including research in the arts) within relevant disciplines; 

• The curriculum ties in with developments in the relevant academie 
discipline(s) by demonstrable links with current academie theories; 

• The curriculum safeguards the development of skills in the field of 
research and/or the development and practice of the arts; 



• When appropriate, the curriculum has demonstrable links with the current 
practice of relevant professions. 



Correspondence 


The curriculum, the teaching concept, the teaching methods and the test 


between aims and 


formats reflect the learning outcomes to be attained by the students. 


objectives and 




curriculum 


The learning outcomes to be attained have demonstrably been translated into 


learning aims and objectives for (components of) the curriculum. 


Cohesion in 


The contents of the curriculum are coherent. 


curriculum 




Duration 


The course duration is in keeping with the statutory standards. 


Admission 


The form and content of the curriculum are geared to the qualifications of 


requirements 


incoming students. 4 



Bachelor's programmes: 

• secondary education qualifications, certificate of short higher education 
courses with full curriculum, certificate of higher education programmes 
aimed at social promotion or a diploma or certificate recognised as 
equivalent under a law, decree, European directive or other international 
agreement; 

• conditions determined by the board of the institution for applicants that do 
not meet the above conditions. 

Advanced bachelor's programmes: 

• bachelor's degree, with (a) particular qualification(s) determined by the 
board of the institution, possibly supplemented by a study into the 
candidate's aptitude or competence or a preparatory curriculum. 



"The admission requirements outlined are those laid down 
in the Higher Education Act. With effect from 2005-2006, 
the requirements for admission to higher education have 
been regulated by the Flexibilisation Act. In terms of 
content, these admission requirements are identical to the 
ones listed here. 
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Master's programmes: 

• bachelor's degree, with (a) particular qualification(s) determined by the 
board of the institution and, when appropriate, supplemented by an 
individualised curriculum, a preparatory curriculum or bridging curriculum. 

Advanced master's programmes: 

• master's degree, with (a) particular qualification(s) determined by the 
board of the institution, possibly supplemented by a study into the 
candidate's aptitude or competence or a preparatory curriculum. 



Number of credits 


The programme meets formal requirements regarding the number of credits: 




• Bachelor's programmes: a minimum of 180 credits 




• Advanced bachelor's programmes: a minimum of 60 credits 




• Master's programmes: a minimum of 60 credits 




• Advanced master's programmes: a minimum of 60 credits 


Master's thesis 


Master's programmes are rounded off with a master's thesis. This carries at 




least one fifth of the total number of credits, with a minimum of 1 5 and a 




maximum of 30 credits. 


Staff commitment 




Standards 


Criteria 


Requirements 


The programme is in keeping with the following criteria for the staff 


regarding 


commitment for programmes with a professional or academie orientation: 


professional / 




academie orientation 


Programmes with a professional orientation: 




• A major part of the curriculum will be taught by staff who establish a link 




between the programme and professional or arts practice. 




Programmes with an academie orientation: 




• A major part of the curriculum will be taught by staff who contribute to the 




development of the field of study (including research in the arts). 




• In addition, when appropriate, the programme will have sufficiënt staff 




with knowledge of and insight into relevant professional or arts practice. 


Quantity of staff 


The capacity made available is sufficiënt to launch the new programme. 




The capacity made available is sufficiënt to continue the new programme. 


Quality of staff 


The staff that will be asked to implement the programme are qualified for the 




substantive, educational and organisational realisation of the curriculum. 


Facilities 




Standards 


Criteria 


Material facilities 


The proposed accommodation and material facilities are sufficiënt to realise 




the curriculum. 


Tutoring 


The staff capacity arranged for tutoring and information provision to students 




is sufficiënt to safeguard their progress. 
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2.5 Internal quality assurance 



Standards 


Criteria 


Systematic approach 


An internal quality assurance system is in place, involving measures for 




improvement taken on the basis of testable targets and periodical evaluations. 


Involving staff, 


Staff, students, alumni and the relevant professional field will be actively 


students, alumni and 


involved in the programme's internal quality assurance. 


the professional field 




Conditions for continuity 


Standards 


Criteria 


Graduation 


The institution provides its students with the guarantee that they can complete 


guarantee 


the curriculum in its entirety. 


Investments 


The investments envisaged are sufficiënt to establish the programme 




(including the facilities). 


Financial facilities 


The financial facilities are sufficiënt to be able to offer the full course of the 




programme. 
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3 Assessment rules 

The proposal for a new programme is assessed by the accreditation organisation. This 
assessment determines whether the new programme provides potentially sufficiënt generic 
quality standards. Each of the standards is evaluated with either a satisfactory or an 
unsatisfactory result. For a positive assessment, each theme in the assessment framework 
must score a satisfactory. The individual theme scores are determined by weighing the 
conclusions regarding the various standards in the theme under consideration. 
Transparency is provided with respect to the manner in which the assessment of the various 
standards has resulted in the summarising conclusion on a theme, in other words, how - 
given the criteria in this assessment framework - the organisation arrived at its conclusion 
for each theme on the basis of the analysis of each Standard. 

In its final conclusion regarding the new programme, the accreditation organisation will 
indicate how this conclusion is based on the facts, the analysis of these facts and the 
assessment of the programme based on this assessment framework. 

The essence of these assessment rules has been laid down in the regulations pertaining to 
"the determination of administrative principles that bear on the decision-making regarding 
the accreditation and initial accreditation of programmes provided in the Flemish 
Community". Articles 7 and 8 of these regulations read: 

"Art. 7. An accreditation decision or initial accreditation report, as the case may be, is 
positive if all summarising conclusions on the themes listed in the Accreditation Framework 
or the Assessment Framework, as the case may be, are satisfactory. If, at the discretion of 
NVAO, standards judged as "unsatisfactory" are compensated by strengths within other 
standards under that same theme, a theme will be judged as "satisfactory". 
Art. 8. In the event of programmes comprising various specialisations and/or locations, a 
theme can only score a "satisfactory" if each of the various specialisations and/or locations 
is judged as "satisfactory". 
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4 Initial accreditation procedure 

1 . The points of departure for the assessment are the Flemish Higher Education Act, the 
above regulations and the themes, standards, criteria and assessment rules laid down in 
this assessment framework. 

2. For each new programme, the institution draws up documents that portray the proposed 
new programme and demonstrate that the programme meets the statutory requirements. 
In addition, the institution will provide the following information: 

- a description of the programme on the basis of the themes and standards of the 
assessment framework; 

- a financial overview stating the expenditure relating to the realisation of the 
programme; 

- a description of the required staff, stating numbers and qualifications. 

3. The institution submits an application for initial accreditation to the accreditation 
organisation, enclosing the information listed under 2. The format and content of the 
application dossier must conform to the regulations laid down by the accreditation 
organisation. For programmes provided by statutory registered institutions 5 , this 
application can only be submitted after a positive judgement by the Recognition 
Committee regarding the macro efficiency of the programme or a positive judgement to 
that effect by the Flemish government; a judgement of the Flemish government is 
deemed positive if not communicated within thirty calendar days. The application must be 
submitted within a term of fifteen days after receipt of the judgement referred to above or 
after expiry of the period of time within which the Flemish government is to pass its 
judgement. 

The application must state the name of the institution and, if appropriate, the association it 
belongs to. In addition, the application must indicate: 

- the name of the programme; 

- the field of study or the (components of) fields of study; 

- whether the programme already exists within the Flemish higher education system or 

- whether this is a programme that is new to the higher education system in Flanders; 

- whether the proposed programme trains students for a bachelor's degree with a 
professional orientation, a bachelor's degree with an academie orientation, a master's 
degree, an advanced bachelor's degree or an advanced master's degree; 

- which specialisations the new programme comprises; 

- whether the programme is to be provided at one or more locations. 

4. On the basis of the data listed under 3 and the initial accreditation procedure it has laid 
down, the accreditation organisation decides the scope of the assessment, the procedure 
to be followed and the expertise to be commanded by the designated external experts. 

5. The actual assessment is conducted by an external panel of experts commissioned by 
the accreditation organisation. They will lay down their findings in an advisory report, in 
accordance with the regulations pertaining to the initial accreditation methodology. 



5 Article 7 of the Flemish Higher Education Act defines 
statutory registered institutions as: the university colleges, 
the universities, the institutions providing post-graduate 
programmes and the recognised faculties for Protestant 
theology. 



Page 10 NVAO I September 2009 



nederlands- vlaamse accreditatie organisatie 



6. In their assessment, the panel experts determine whether the plans and documents meet 
the criteria laid down in this assessment framework. The assessment results in a 
summarising conclusion regarding the new programme, which is motivated in an initial 
accreditation report. 

7. On the basis of the advisory report, the accreditation organisation draws up its initial 
accreditation report, which contains a summarising conclusion. This initial accreditation 
report must meet the following criteria: 

a. The quality score awarded in the assessment is - insofar as relevant - also based on a 
comparison with other, similar programmes and internationally accepted criteria tor 
programmes in the domain under consideration. 

b. The initial accreditation report indicates whether or not the new programme potentially 
provides sufficiënt generic quality standards. The report must discuss at least the six 
themes listed in this assessment framework, devoting attention to all the standards 
under each theme. For each Standard, a satisfactory or unsatisfactory valuation is 
awarded, which serves as the basis for the conclusion given for each theme. The 
conclusions are underpinned by facts and analyses wherever possible. The report is 
finalised with a summarising conclusion regarding the new programme. 

c. The initial accreditation report describes the procedure followed in the assessment. It 
explicitly explains: 

- the methods used; 

- the sources of information used; 

- the reference framework used in the assessment. 

The accreditation organisation draws up its initial accreditation report and summarising 
conclusion on the basis of the advisory report, once it has ascertained that the advisory 
report meets the three criteria referred to above and that the methods, sources of 
information and reference framework used are reliable. 

8. The accreditation organisation will make a decision within four months after receipt of the 
application. The decision is laid down in an initial accreditation report. Before the 
timeframe of four months expires and before finalising the initial accreditation report, the 
accreditation organisation allows the board of the institution a term of ten days to 
formulate appeals and comments regarding the draft initial accreditation report. If the 
accreditation organisation deerns that the generic quality standards provided for the new 
programme are potentially sufficiënt, the assessment results in a positive initial 
accreditation report. 

The accreditation organisation forwards its initial accreditation report to the institution and 
the Flemish minister responsible for education. 



Explanatory notes 

General 

New programmes 

A new programme is a programme that does not ensue from an accepted transformation of 
the existing programmes on offer and that is not registered to the institution involved on the 
Higher Education Register. Two types of new programme can be distinguished: 
programmes that are entirely new to the Flemish higher education system and that are not 
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yet listed on the Higher Education Register and programmes that are new to the institution 
involved but nonetheless registered on the Higher Education Register. 

The Flemish government decides on the recognition of new programmes. To that end, the 
programmes must meet the conditions stated in Articles 61 and 62 of the Higher Education 
Act, such as passing the initial accreditation by the accreditation organisation and a positive 
judgement from the Recognition Committee or the Flemish government with respect to the 
macro efficiency of the programme. The initial accreditation determines whether the generic 
quality standards are potentially represented to a sufficiënt degree in the new programme. 

A newly recognised programme is deemed to have been accredited up to and including the 
end of the second academie year in which the first cohort of students completed the full 
study load laid down for the new programme. 

Subject to assessment by the accreditation organisation 

One of the tasks of the accreditation organisation is to assess whether applications for new 
programmes demonstrate sufficiënt potential for generic quality standards. This assessment 
also answers the question of whether the programme meets the necessary conditions for 
launching a new programme and whether the institution is capable of providing the new 
programme in a sustainable manner. 

A positive initial accreditation decision from the accreditation organisation is one of the 
conditions to be met in order to be able to offer a new programme. With a view to the 
commensurable legal effects - the authorisation to offer a recognised programme - the 
themes, standards and criteria observed in initial accreditations must in essence correspond 
to those used in the accreditation of existing programmes. At the standards and criteria 
level, however, the accreditation of existing programmes differs from initial accreditations. 
Unlike the assessment of existing programmes, the assessment of new programmes 
involves an assessment based on a plan. The question is whether the proposal is 
underpinned by clear aims and objectives that tie in with the various requirements set for 
bachelor's programmes with a professional orientation, those for bachelor's and master's 
programmes with an academie orientation and the subject-specific requirements. In 
addition, the plan must comprise a description of the curriculum and the facilities that 
justifies the expectation that the aims and objectives will be achieved. In order to be able to 
assess these themes, high requirements are set for the quality of the information to be 
provided by the institutions (burden of proof principle). The accreditation organisation calls 
in external experts for the assessment. 

A key point of departure in the assessment of new programmes is that the scope of the 
external assessment may depend on the extent to which a proposed new programme differs 
from already existing programmes, on the degree of newness of the programme to the 
institution or to Flanders, and on the elements that are already in place (already existing 
programme components, staff, infrastructure, financial resources) for the realisation of the 
new programme. Applications that differ substantially from existing programmes in terms of 
content will be subjected to a more extensive assessment than applications for programmes 
that are already offered elsewhere in the Flemish higher education system. Thus, the 
assessments may differ as regards depth of investigation. For each application, the actual 
set-up of the assessment procedure is determined on the basis of the procedure regulations 
laid down. 

The assessment pertains to the expected quality of the programme. 

If a new programme involves various specialisations or locations, the assessment must 
demonstrate that the generic quality is safeguarded for the entire programme with all its 
specialisations and at all its locations. 
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Transparency 

The accreditation organisation regards it as its duty to contribute to the transparency of the 
programmes on offer. This means that it will ascertain whether each proposed new 
programme is sailing under true colours, i.e., whether the aims and objectives and the 
curriculum are in keeping with the proposed nomenclature. For proposed new programmes 
whose nomenclature is already listed on the Higher Education Register, it will ascertain 
whether the aims and objectives and the curriculum sufficiently correspond to existing 
programmes registered on the Higher Education Register under that same nomenclature, in 
order to justify the nomenclature under consideration for that programme. With regard to 
new programmes that are not yet offered elsewhere in Flanders under that nomenclature 
and must therefore be regarded as truly new in terms of content, the institution will have to 
satisfy the accreditation organisation that the proposed programme differs substantially from 
existing programmes and consequently must not be regarded as a variation of such 
programmes. 

Assessment f ramework 

This assessment framework is based on the Flemish Higher Education Act of 4 April 2003 
and the Assessment Framework for New Programmes established by the Dutch 
Accreditation Organisation (NAO) on 14 February 2003. We aimed for maximum 
harmonisation between the Dutch and the Flemish frameworks. We deviated from the Dutch 
framework whenever the Flemish Higher Education Act or differences between the Flemish 
and Dutch higher education systems so necessitated. For various aspects, these 
explanatory notes explicitly highlight the connection with the regulations governing higher 
education in Flanders and explain the themes, standards and criteria within the context of 
the Flemish situation. 



The assessment of new programmes is based on evaluation of the following themes: 

- aims and objectives of the programme; 

- curriculum; 

- staff commitment; 

- facilities; 

- internal quality assurance; 

- conditions for continuity. 

These themes have been subdivided into standards. The table below indicates how the 
themes and standards of the assessment framework tie in with the generic quality standards 
from the Flemish Higher Education Act (Article 58). 



Generic quality standards, 
Higher Education Act 



Assessment framework for new programmes in 
Flanders 
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Study contents: 

- nature and educational level 

- cohesion in curriculum 

- workload 

- relationship between aims and 
objectives and content 



Teaching and learning process: 

- harmonisation of format and content 

- tutoring 

transparent assessment and testing 



Educational outcomes: 

- social relevance 
learning outcomes 

- success rates 

Material facilities, quality of staff, 
organisation and internal quality assurance 



Self-evaluation methods 



Aims and objectives: 

- level and orientation, bachelor's degree 
professional higher education 

- level and orientation, bachelor's degree 
academie education 

- level and orientation master's degree 

- domain-specific requirements 

Curriculum: 

- requirements for professional and academie 
orientations 

- relationship between aims and objectives and 
content 

- cohesion in curriculum 

- workload 

- admission requirements 

- number of credits 

Curriculum: 

- relationship between aims and objectives and 
curriculum 

- master's thesis 

Services and facilities: 

- tutoring 

(related to macro efficiency check) 



Staff commitment: 

- requirements for professional/academie orientations 

- quantity of staff 

- quality of staff 

Services and facilities: 

- material facilities 

Conditions for continuity: 

- graduation guarantee 

- investments 

- financial facilities 

Internal quality assurance: 

- systematic approach 

- involvement of staff, students, alumni and 
professional field 



n.a. 



Several elements of the generic quality standards - transparent assessment and testing, 
educational outcomes, self-evaluation methods, harmonisation of format and content - have 
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not or only partially been included in the criteria for this assessment framework because 
these elements do not bear an ex ante assessment or are fairly irrelevant for the 
assessment of plans. The additional theme of conditions for continuity has been added in 
order to ensure that the new programme can actually be offered. 

As far as possible, the structure of the assessment framework, the choice of themes, 
standards and criteria and the level of abstraction observed are in keeping with the Dutch 
assessment framework. 

The point of departure is that the institution, within its authorisation to provide education, 
submits an application for a new programme as a bachelor's programme with a professional 
orientation, a bachelor's programme with an academie orientation, a master's programme, 
an advanced bachelor's programme or an advanced master's programme. For that reason, 
the programme must be assessed as such. 

An assessment of a new programme is aimed at reviewing, on the basis of the themes, 
standards and criteria of the assessment framework, whether the accreditation organisation 
is justified in concluding that the programme has sufficiënt generic quality standards in 
place. To that end, it may review the plans and finalised curricula but also existing elements 
that will be used in the new programme (staff, material resources, courses or components of 
existing programmes that will be incorporated into the new programme...). This does not 
mean, however, that the quality of achievements or results ensuing from these existing 
elements will be judged by reference to the existing programme(s), but rather that they will 
be taken into consideration from the point of view of quality guarantees for the new 
programme. The presence of these quality guarantees is always assessed on the basis of 
the assessment framework for new programmes, not the assessment framework for existing 
programmes, even if certain components of the proposed new programme already exist. 

To some extent, the assessment of new programmes imposes different requirements on the 
expertise of those conducting the assessment than an assessment of existing programmes. 
Particular differences relate to the ex ante assessment of the quality of the programme, the 
quality of the staff to be committed and the business aspects. 



Domain-specific requirements 

A programme assessment cannot be based solely on general criteria pertaining to aims and 
objectives, curriculum, services and facilities, staff commitment and internal quality 
assurance. In its description, the institution must take account of the requirements set for 
this specific programme by (foreign) peers and professional or art practice. With regard to 
regulated professions, the relevant rules and regulations must also be taken into account. 

Embedding academie programmes in research 

Essential characteristics of programmes with an academie orientation are the fact that these 
programmes are inseparable from research and the fact that teaching and research are 
intertwined. Teaching in programmes with an academie orientation must be founded on 
research. Various paragraphs of the assessment framework comprise requirements relating 
to the inseparability of research and teaching. These requirements pertain to: 
- the aims and objectives of the programmes (in accordance with the stipulations contained 
in Article 58 of the Higher Education Act); 



6 The domain-specific requirements operationalise the 
generic quality standards. They are subordinate to these 
standards in terms of hierarchy. 
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- the curriculum (requirements regarding academie orientation, such as interaction between 
education and research, keeping abreast of academie developments and current 
academie theories, guaranteeing skills in the field of research and the master's thesis); 

- staff commitment (teaching is principally provided by researchers who contribute to the 
development of their field of study). 

These requirements pertain to all programmes with an academie orientation. Their 
assessment must also be based on the research activities of the staff (such as research 
projects, doctoral research and academie output). The actual interpretation of the 
requirements will vary according to the nature of the programme (bachelor's, master's or 
advanced master's) and their position in the course of the programme. As a rule, master's 
programmes will tend to be more firmly embedded in research than bachelor's programmes 
and the introduction of research skills will feature primarily in master's programmes, most 
prominently in the master's thesis. Depending on the domain-specific nature of the 
programme, the requirements to be met by the programmes will be specified in more detail 
(cf. the explanatory note to Chapter 2, the curriculum). 

Scope 

A positive result in the initial accreditation is one of the conditions for new programmes, 
whether provided by statutory registered institutions or by non-statutory registered 
institutions, to be recognised and included on the Higher Education Register. Registration of 
the programme on the Higher Education Register is a requirement for the authority to award 
recognised bachelor's and master's degrees. Underthe Higher Education Act, the possibility 
is created for other institutions to be registered, offer recognised bachelor's and master's 
programmes and award the protected bachelor's and master's degrees. New programmes 
that are offered by such registered institutions must also pass the initial accreditation. 

Advanced bachelor's and advanced master's programmes 

The entrance requirements for advanced bachelor's and advanced master's programmes 
differ from those that apply for regular bachelor's and master's programmes. Otherwise, 
these programmes must meet the same quality requirements as other bachelor's and 
master's programmes. 
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Explanatory notes, chapter by chapter 

Chapter 1 : Structure of the assessment framework 

The assessment framework comprises substantive criteria and criteria for the procedures to 
be followed with regard to the assessment and the initial accreditation report. Unlike the 
assessment of existing programmes, the assessment of new programmes does not offer the 
additional opportunity to review distinctive features. Our point of departure is that such 
features must first have demonstrated their worth in actual practice before they can be 
judged. 

Chapter 2: Evaluation framework 

The accreditation organisation reviews the assessment of the new programme on the basis 
of the criteria laid down in the assessment framework. These criteria refer to the statutory 
generic quality standards. The object of the assessment is to ascertain whether potentially 
sufficiënt generic quality standards are in place. 

- Re 2.1 : Aims and objectives of the programme 

The aims and objectives of the programme refer to the statutory competencies students 
must acquire in the intended programmes. 

The aims and objectives of the programme must be geared to the domain-specific quality 
requirements and general criteria regarding the level and orientation of bachelor's 
programmes with a professional orientation and bachelor's / master's programmes with an 
academie orientation. In addition, the aims and objectives of the programme must reflect the 
intention to keep abreast of recent developments in the discipline and, as the occasion 
arises, professional practice. 

- Re 2.2: Curriculum 

The aims and objectives pursued must be convincingly concretised in the curriculum of the 
programme. In addition, the proposed contents and format of that curriculum (including the 
attainment targets) must enable qualified entering students to acquire the envisaged 
competencies within the stipulated timeframe. A review of the translation of the aims and 
objectives into the curriculum requires a substantive review of the study package and the 
educational set-up, given the specific choice for bachelor's and master's programmes with 
either a professional or an academie orientation and the domain in question. Compared to 
bachelor's programmes, a master's programme will be characterised by a more in-depth 
and/or broader approach. Programmes with a professional orientation will tend to focus on 
the introduction of knowledge and experience from current professional practice, whereas 
embedding in research and developing research skills constitute essential aspects of 
programmes with an academie orientation. Some academie programmes, however, have an 
additional professional orientation. For that reason, the requirements for academie 
bachelor's and master's programmes also stipulate that the curricula of eligible programmes 
must have demonstrable ties with current practice in relevant professions. 
In order to be able to review the programmes in this context, those who conduct the 
assessment will usually have to command relevant domain-specific expertise. 

Depending on the domain-specific contents of the master's programmes, higher 
requirements will be set for certain standards. For example, for master's programmes with a 
particular focus on researcher training, the domain-specific requirements will need to tie in 
with international standards for such programmes; in the review of the aims and objectives 
of these programmes, the emphasis will be placed on mastering the competencies required 
for independently conducting research at a newly-qualified researcher's level; and the 
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embedding of these programmes in research will have to meet more stringent requirements. 
This means that higher requirements will be set for, among other things, the research 
capacity of the staff and their experience in training newly-qualified researchers, a broad 
research orientation in the curriculum, the academie interpretation of the master's thesis and 
the connection with current academie developments. 

As applied academie research is also one of the tasks of university colleges, (applied) 
research has been included in the options for knowledge development for bachelor's 
programmes with a professional orientation. 

A plan for a new programme will need to explain in clear terms how the new programme will 
be organised. Attention must be paid to the cohesion between the aims and objectives and 
the contents of the curriculum, the cohesion within the curriculum, the workload and the 
manner in which the programme is tailored to the enrolling students. The plan must sketch a 
realistic picture of the practice of the intended programme. The wording of the assessment 
framework, on the basis of which the curriculum is reviewed, leaves sufficiënt room for 
current developments with regard to the standards referred to above. 

The quality of the curriculum will also have to be demonstrated by the manner in which the 
curriculum is geared to the qualifications of enrolling students. Important considerations in 
this respect are the statutory requirements for bachelor's and master's programmes, as 
outlined in the assessment framework. 

Whereas the programme curriculum is reviewed as a model course for the realisation of its 
aims and objectives, on the students' side the introduction of the Flexibilisation of Higher 
Education Act in Flanders has created opportunities for individualised training routes, 
flexible curricula, learning environments, educational organisation and the recognition of 
prior learning. As the occasion arises, these forms of flexibilisation, students' careers 
guidance and their study progress counselling will be taken into consideration in the review 
of the themes, standards and criteria involved. 

- Ad 2.3: Staff commitment 

The commitment of staff is an important condition for quality and decisive for the distinction 
between professional and academie orientations. A significant part of programmes with a 
professional orientation will be taught by staff who establish a connection between the 
programme and professional or arts practice. Programmes with an academie orientation, on 
the other hand, will largely be taught by researchers who contribute to the development of 
the discipline (including research in the arts). In addition, the appropriate academie 
programmes will have sufficiënt staff with knowledge of and insight into the relevant 
professional or arts practice. 

The institution must indicate the number of staff to be committed to the new programme, the 
availability of such staff and the quality of the intended staff. The review of the intended staff 
commitment to the programme weighs heavily, as the quality of the new programme will be 
directly determined by the available staff. A separate consideration is whether this 
commitment suffices for launching the programme - which will require an additional effort - 
and for its continuation on a regular basis. The assessment will also have to review whether 
the staff designated by the institution as deployable will indeed be available for the new 
programme. 
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The Higher Education Act does not explicitly mention the Standard of staff quantity among 
the generic quality standards. 7 This Standard has been included in the assessment 
framework because the presence of sufficiënt staff is an evident condition for generic 
quality. 

- Re 2.4: Services and facilities 

The criteria that apply to the services and facilities are, in principle, identical to those 
observed in the accreditation of existing programmes. The services and facilities must be 
sufficiënt to realise the curriculum. The institution must make a reasonable case that the 
services and facilities concerned will be available (in time). 

The services and facilities will likely include, for example, media centres, laboratories, 
educational equipment, studio space or subject-specific data bases. The nature and level of 
these services and facilities differ depending on the nature of the programme. The 
assessment will check whether the services and facilities for the new programmes are in 
accordance with the generally received level of quality for such services and facilities. 

- Re 2.5: Interna! quality assurance 

The institution must demonstrate that it intends to pursue an acceptable level of quality from 
the outset of the new programme and will subsequently continuously seek to monitor and 
improve that quality. These intentions are more likely if the institution as a whole has in 
place an effective system of internal quality assurance that covers all its programmes. 

- Re 2.6: Conditions for continuity 

One of the objects assessed with regard to new programmes is the uncertainties involved in 
a starting situation, in particular the question of whether the administrative and staff 
conditions warrant the sustainable realisation of the intended substantive quality. These 
conditions pertain to both the programme as such and the institution providing it. It must be 
likely that the institution can raise the required initial investment and continue to run the 
programme, in the longer term as well, at least for the number of years required to build up 
a full curriculum and the corresponding population of enrolled students. In principle, this 
period of time equals the envisaged duration of the curriculum, depending on the nature of 
the programme (bachelor's programmes with a professional orientation, bachelor's and 
master's programmes with an academie orientation, advanced bachelor's programmes, 
advanced master's programmes). As a rule, this will be demonstrated by a business plan for 
the new programme. The financial strength of the institution as a whole is demonstrated by 
its annual accounts. 

It is advisable that an application for a new programme is accompanied by a plan that 
shows which programmes the institution intends to offer for the medium range. In this plan, 
which will usually form part of a long-range plan or a strategie memorandum, the institution 
can make a reasonable case that: 

- the intended new programme is an expression of the institution's strategie policy and the 
ensuing priorities in the programmes offered; 

- it will be capable, in terms of business economics, of providing the total range of existing 
and intended programmes in a permanent manner. 

The above plan can also serve as source of information for assessing whether the intended 
investments and the envisaged operation of the new programme are realistic in the context 
of the overall plans and expenditure of the institution. Submitting such a plan is not 
mandatory on the part of the institution but can support the assessment process. 

7 The Dutch legislation does not explicitly mention this 
Standard either. 
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To the accreditation organisation, the data concerned is of importance in the longer term as 
well. In case of a subsequent application for accreditation, the accreditation organisation will 
ascertain or have ascertained whether the institution has fulfilled the conditions for continuity 
and acted consistently in line with the intentions stated in its application for recognition. 

Chapter 3: Assessment rules 

The assessment results in a summarising review of the potential quality of the new 
programme, with a positive or negative overall conclusion regarding the presence of 
sufficiënt generic quality standards. A positive final conclusion requires a satisfactory score 
on all themes of the assessment framework. For each theme, the conclusion is based on a 
review of the various standards contained in that theme. The initial accreditation report gives 
a transparent overview of the considerations at all levels - standards, themes and final 
conclusion - which provides clarity regarding the underpinning of the final conclusion and 
the weighing-up of the various standards. 

Programmes may comprise more than one specialisation. If these specialisations set 
different requirements with regard to a number of quality aspects - such as cohesion, 
tutoring, staff commitment and tailoring to entering students - the review must demonstrate 
that sufficiënt generic quality standards are potentially in place for each specialisation. The 
assessment, however, concerns the programme as a whole. A similar assessment rule 
applies to new programmes that are provided at more than one location. 

Chapter 4: Initial accreditation procedure 

In its assessment of new programmes, the accreditation organisation ties in with the 
requirements and timeframes laid down in the Flemish Higher Education Act and the 
regulations regarding the methodology for initial accreditations. An essential feature of this 
method of working is the initiating role of the institution. It determines the nature of the 
programme (bachelor's programme with a professional orientation, bachelor's programme 
with an academie orientation, advanced bachelor's programme, master's programme, 
advanced master's programme). With regard to programmes provided by statutory 
registered institutions, applications may only be submitted following a positive conclusion by 
the Recognition Committee or an (explicit or implicit) positive conclusion by the Flemish 
government regarding their macro-efficiency. Submitting an application to the accreditation 
organisation sets the assessment process in motion. Based on the application, the 
accreditation organisation determines how new this programme is to the Flemish higher 
education system and consequently how extensively it needs to be assessed. 
Subsequently, the accreditation organisation commissions external experts to review the 
application and draw up an advisory report. 

On the basis of the advisory report, the accreditation organisation then draws up an initial 
accreditation report with its summarising conclusion. Guiding elements in the formulation of 
the report are the benchmarks outlined in this chapter. The conclusions in the initial 
accreditation report are substantiated on the basis of facts established, which are analysed 
and tested against a reference framework that ties in with the assessment framework for 
new programmes. 

The initial accreditation report also explains the methods, sources of information and 
reference framework used for the assessment. With regard to programmes new to the 
Flemish education system, it indicates how the domain-specific requirements have been 
mapped out. For existing programmes, it indicates which programmes have been used as a 
reference. 

The initial accreditation report gives an account of the procedure followed. 
Before finalising the initial accreditation report, the accreditation organisation allows the 
board of the institution a term of ten days to present its view regarding the draft initial 
accreditation report. The initial accreditation report is positive if, in the opinion of the 
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accreditation organisation, the application gives reasonable cause to assume that the new 
programme will pass the test regarding the presence of sufficiënt generic quality standards. 
The accreditation organisation must explicitly indicate whether or not the initial accreditation 
is positive. 

Following its finalisation, the accreditation organisation submits the initial accreditation 
report to the board of the institution and the Flemish minister responsible for higher 
education. 
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(Nederland) 
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Code of Good Practice, European Consortium for Accreditation in higher education, Zürich, 3 december 2004. 

Communicatieplan 201 - 201 1 , NVAO, Den Haag, 2 juni 201 0. (Nederland) 

Communicatieplan Vlaanderen, 16 november 2010 (Vlaanderen) 

Decreet betreffende de herstructurering van het hoger onderwijs in Vlaanderen, 4 april 2003 en latere versies. 
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Evaluatieve gesprekken in Vlaanderen: thematische samenvatting van de bevindingen en overwegingen van de 
NVAO, 21 oktober 2011. (Vlaanderen) 

Frameworks for the Assessment of Internationalisation, 14 November 201 1 . 

Handleiding accreditatie Vlaamse Gemeenschap, 1 september 2009. (Vlaanderen) 

Handleiding Uitschrijven Domeinspecifieke Leerresultatenkaders, proefversie voor de pilot, zonder datum. 
(Vlaanderen) 

Handleiding erkenning evaluatieorganen, 22 februari 2005. (Vlaanderen) 

Handleiding toets nieuwe opleiding Vlaamse Gemeenschap, 1 september 2009. (Vlaanderen) 

Handleiding onderwijsvisitaties VLIR en VLHORA, Brussel, September 2008 

Handreiking TNO Panel: samenstelling, taakstelling en werkwijze, 22 augustus 201 1 . 

Handreiking aanvraag Toets Nieuwe Opleiding Nederland, 21 oktober 201 1 . (Nederland) 

Jaarboek onderwijs in cijfers 2009, 2010 en 201 1 , Centraal Bureau voor de Statistiek, Voorburg / Heerlen, 
2009/2010/2011. 

Kaders HB05, 9 februari 2010. (Vlaanderen) 

Kwaliteit in verscheidenheid, Strategische agenda hoger onderwijs, onderzoek en wetenschap, Ministerie van 
OCW, Den Haag, augustus 201 1 . (Nederland) 

Leereffectenrapport proefprojecten HB05, NVAO, December 2010. (Vlaanderen) 
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Protocol cursusduur masters, 8 oktober 2003. (Nederland) 

Protocol erkenning als 'rechtspersoon voor hoger onderwijs', augustus 2010. (Nederland) 

Protocol verlengde masters Beoordeling van de verlenging van de cursusduur van enkele specifieke 
masteropleidingen in het WO, 23 april 2003. (Nederland) 
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7 september 2010. (Nederland) 

Toetsingskader nieuwe opleidingen hoger onderwijs Vlaanderen, 1 september 2009. (Vlaanderen) 
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domeinspecifieke leerresultaten van hogeronderwijsopleidingen in de Vlaamse Gemeenschap, 1 februari 201 1 . 
(Vlaanderen) 

Reglement betreffende vormvereisten voor aanvragen tot uitvoering van de accreditatie of de Toets nieuwe 
opleiding ten aanzien van hogeronderwijsopleidingen in de Vlaamse Gemeenschap, 1 september 2009. 
(Vlaanderen) 

Samenwerkingsprotocol NVAO - Inspectie voor het hoger onderwijs, 30 januari 2008. (Nederland) 

Standpunt NVAO ten aanzien de Masterproef in de kunstopleidingen Vlaanderen, November 2009. (Vlaanderen) 

Standards and Guidelines for Quality Assurance in the European Higher Education Area, European Association 
for Quality Assurance in Higher Education, Helsinki, 2005. 

Strategie NVAO 201 2 - 201 6, Den Haag, 1 3 februari 201 2. 

Verdrag tussen het Koninkrijk der Nederlanden en de Vlaamse Gemeenschap van België inzake de accreditatie 
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Attachment 13 List of Abbreviations 



AKOV 


Agentschap Kwaliteit Onderwijs en Vorming 


ANECA 


Agencia Nacional de Evaluación de la Calidad y Acreditación 


AQUIN 


Akkreditierungs-, Certifizierungs- und Qualitatssicherungs-lnstitut für alle Studiengange 


Awb 


Algemene wet bestuursrecht 


CROHO 


Centraal Register Opleidingen Hoger Onderwijs 


DLR 


Domeinspecifieke Leerresultaten 


DUO 


Dienst Uitvoering Onderwijs 


ECA 


European Consortium tor Accreditation in higher education 


EFQM 


European Framework of Quality Management 


ENQA 


European Association for Quality Assurance in Higher Education 


E-Train 


The project European Training of Quality Assurance experts 


EQAR 


European Quality Assurance Register for Higher Education 


EQF 


European Qualifications Framework 


EVA 


Danish Evaluation Institute 


HAVO 


Hoger Algemeen Voortgezet Onderwijs 


hbo 


hoger beroepsonderwijs 


HBO-raad 


Vereniging van hogescholen (Nederland) 


HB05 


Hoger beroepsonderwijs op niveau 5 van de Vlaamse kwalificatiestructuur 


HOR 


Hogeronderwijsregister 


IQA 


Internal Qulity Assurance 


JOQAR 


Joint programmes: Quality Assurance and Recognition of degrees awarded. 


INQAAHE 


International Network for Quality Assurance Agencies in Higher Education 


MBO 


Middelbaar beroepsonderwijs 


NLQF 


Netherlands Qualificatons Framework 


NQF 


National Qualifications Framework 


NVAO 


Nederlands-Vlaamse Accreditatieorganisatie 


OCW 


Ministerie van Onderwijs, Cultuur en Wetenschappen (Nederland) 


OECD 


Organisation for Economie Co-operation and Development 


OZM 


onderzoeksmasters 


PhD 


Doctor of Philosophy 


QAA 


Quality Assurance Agencies 


QAA 


Quality Assurance Agency for Higher Education in the United Kingdom 



SERV 


Rnriaal-Fmnnmi^rhp Raad van Vlaandprpn 
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TEAM II 


Transparant EuropGan AccrGditation dGcisions & Mutual rGCognition agrGGmGnts 


TNO 


TnptQ nipiiwp nnlpidinn 

i ucio i iicuïïc vjjjicivjn ly 


UAS 


UnivGrsitiGS of AppliGd SciGncGS 


VLHORA 


Vlaam^p Honp^cholpnraad 


VLIR 


Vlaam^p Intprunivpr^itairp Raad 

V 1 Cl dl 1 IOC III IC 1 UI II VCI iJlLuM C 1 1CIC4.V.1 


VLOR 


Vlaamse Onderwijsraad 


VLUHR 


Vlaamse Universitaire en Hogescholenraad 


VSNU 


Vereniging van Samenwerkende Nederlandse Universiteiten 


VWO 


Voorbereidend Wetenschappelijk Onderwijs 


WO 


wetenschappelijk onderwijs 


WOT 


Wet op het onderwijstoezicht 



